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In  four  ways.  First, ^background  Informatlon^consl sting  of  a  literature  review 
an?  an  analysis  of  selected  accident  reports^as  collected  and  Is  described. 
Second,  £  workshop  was  convened  to  review  the  state-of-the-art  of  aircrew 
emergency  decision  training,  safety  research,  and  behavioral  decision  theory.^ 
The  workshop  resulted  In  the  Identification  of  current  Issues  and  recommcnda-,. 
tlons  for  future  work.  Third, selected  set  of  emergency  'situations  was  the 
basis  of  a  preliminary  classification  of  aircraft  emergency  situations  In  terms 
of  several  situational  and  decision  making  attributes.  The  classification  Is 
based  on  data  derived  from  Interviews  with  experienced  military  flying  per¬ 
sonnel.  Fourth, >ji  taxonoey  of  emergency  situation  types  was  developed,  Incor¬ 
porating  both  situational  and  task  specific  elements  as  cognitive  attributes 
of  the  decision  tasks  performed  under  emergency  conditions.-  There  were  several 
steps  preceding  the  development  of  the  taxonoiqy.  The  aggregation  of  situations 
which  could  be  considered  within  an  emergency  training  program  was  reviewed. 

A  definition  of  the  emergency  situation  was  developed,  which  limited  the  scope 
of  consideration  to  a  manageable  entity--known  malfunctions.  Representational 
models  of  the  objective  (external)  emergency  situation,  decision  processes, 
and  cognitive  functioning  were  proposed  as  a  way  of  characterizing  the  situa¬ 
tional  and  behavioral  aspects  of  an  emergency  malfunction.  The  taxonomic 
structure  was  then  derived  after  consideration  of  the  cognitive  elements  of  the 
three  representational  models. 

On  the  basis  of  the  taxonomy,  three  classes  of  emergency  situations  were  found 
to  be  of  Interest:  Situation  1  (predictable),  Situation  2  (partly  predictable), 
and  Situation  3  (unpredictable).  Initial  training  guidelines  are  suggested  In 
light  of  the  cognitive  requirements  of  each  class. 
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1.  OVERVIEW 


1.1  Objectives  of  Phase  1 

This  report  describes  the  activities  and  findings  of  the  first  phase  of 
a  three-year  research  and  development  effort  to  enhance  pilot  and  air¬ 
crew  emergency  decision  skills.  The  overall  effort  will  provide  a  basis 
for  designing  training  programs  and  training  materials  which  address  the 
development  of  aircrew  emergency  decision  skills  in  a  more  systematic 
and  comprehensive  manner  than  has  been  possible  in  the  past. 

Major  objectives  of  Year  1  were: 

(1)  To  review  the  current  state  of  the  art  of  emergency  deci¬ 
sion  training  for  aircrew  members,  from  both  theoretical 
and  practical  standpoints. 

(2)  To  review  the  decision  research  literature  and  Identify 
methodologies  which  can  be  utilized  to  define  and  analyze 
aircraft  emergency  problems  in  terms  of  cognitive  decision 
functions. 

(3)  To  develop  theoretical  models  of  the  aircraft  emergency 
situation  which  account  for  the  relevant  situational  and 
cognitive  variables  Involved. 

(4)  To  derive  initial  guidelines  from  the  models  which  can  be 
used  in  design  of  training  programs  for  aircrew  emergency 
decision  skills  and  which  facilitate  the  development  of 
training  materials  and  emergency  situation  scenarios. 
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Rationale 


Military  aircraft  accidents  are  important  not  only  to  the  individuals 
directly  involved,  but  also  to  those  responsible  for  preparing  and  main¬ 
taining  combat-ready  forces  for  the  nation's  defense.  The  cost  of  an 
accident  can  be  measured  directly  in  terms  of  injuries,  lives  lost  and 
equipment  repair  and  replacement  costs.  Indirect  costs  which  accompany 
these  accidents  include  the  time  and  training  resources  invested  in  the 
personnel  lost  as  well  as  the  impact  of  the  loss  of  equipment  and 
trained  personnel  on  force  effectiveness. 

The  military  services  have  paid  close  attention  to  the  problem  of  air¬ 
craft  safety  with  the  gratifying  result  that  accident  rates  have  steadi¬ 
ly  exhibited  a  downward  trend.  While  military  aircraft  safety  continues 
to  show  progress,  there  is  still  need  to  press  for  additional  improve¬ 
ments.  Human  error  commonly  contributes  to  approximately  50%  of  mili¬ 
tary  aircraft  accidents  (Nuvolini,  1979)  suggesting  that  emergency  pro¬ 
cedures  and  training  are  a  fruitful  area  in  which  to  expend  additional 
effort. 

An  emergency  is  commonly  defined  as  an  unexpected  occurrence  of  a  set  of 
circumstances  which  calls  for  immediate  judgment  and  action  to  avoid  un¬ 
desirable  consequences.  The  standard  emergency  response  expected  of 
every  aircrew  is  three-fold:  (a)  to  maintain  aircraft  control,  (b)  to 
analyze  the  situation  and  take  proper  action,  and  (c)  to  land  as  scon  as 
practicable.  In  the  broader  context  of  flight  safety,  aircrews  arc  ex¬ 
pected  to  do  more  than  skillfully  resolve  immediate  full-blown  emergen¬ 
cies.  It  is  equally  important  that  they  actively  avoid  situations  trfiich 
might  lead  to  emergencies  and  that  they  recognize  the  early  signs  of  an 
impending  emergency  and  take  corrective  action  before  the  situation  as¬ 
sumes  crisis  proportions. 
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Emergency  preparedness,  according  to  this  vie#,  goes  beyond  the  capabil¬ 
ity  to  make  very  rapid  and  accurate  decisions  under  intense  time  pres¬ 
sure.  It  is  a  truism  among  experienced  flying  personnel  that  there  is 
usually  more  than  enough  time  to  deal  with  most  emergencies  and,  furth¬ 
er,  that  it  is  not  the  first  mistake,  or  even  the  second,  that  kills  pi¬ 
lots,  but  the  third.  An  aircraft  emergency  might  be  viewed,  then,  as  a 
sequence  of  events  and  decisions,  which,  if  not  recognized  and  resolved 
at  an  earlier  stage,  culminate  in  a  crisis.  If  so,  emergency  prepared¬ 
ness  training,  and  more  specifically,  emergency  decision  training,  must 
accommodate  the  broader  range  of  situations  and  skills  that  this  concep¬ 
tion  embraces. 

In  the  strict  sense,  decision  making  can  be  viewed  as  the  efficient 
translation  of  information  into  appropri  v.e  action  by  a  rational  deci¬ 
sion  maker  using  effective  decision  strategies.  While  this  may  be  an 
adequate  description  of  decision  analysis,  a  slightly  broader  view  of 
decision  making  is  necessary  to  encompass  the  decision  activities  impor¬ 
tant  in  the  practical  setting  of  aircrew  emergency  decision  training. 

As  Nickerson  and  Feehrer  (1975)  point  out,  decision  making  Involves  a 
number  of  overlapping  aspects  or  phases  of  activity,  which  might  best  be 
conceived  of  as  a  series  of  related  problem  solving  tasks.  In  our  view, 
the  following  breakdown  is  useful  In  characterizing  the  general  area  of 
decision  making: 

(I)  Situation  Diagnosis. 

(a)  Problan  recognition. 

(b)  Information  acquisition  and  evaluation. 
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(2)  Decision  Making. 


(a)  Problem  structuring  and  development  of  alternatives. 

(b)  Evaluation  of  alternatives  and  selection  of  a  course 
of  action. 

(3)  Decision  Execution. 

(a)  Implementation  of  action  alternative. 

(b)  Monitoring  of  implementation  and  evaluation  of 
resul ts. 

The  impact  of  situational  factors  on  emergencies  and  emergency  decisions 
can  not  be  ignored.  A  decision  which  is  appropriate  under  one  set  of 
circumstances  can  be  fatal  under  another.  Aircrew  members  must  be 
prepared  not  only  to  exercise  the  wide  range  of  decision  skills  suggest¬ 
ed  above,  but  to  consider  relevant  situational  factors  appropriately  in 
formulating  and  executing  emergency  decisions. 

The  present  program  of  research  and  development  considers  the  need  to 
integrate  the  situational  and  decision-making  variables  of  aircraft 
emergencies  into  a  common  cognitive  framework.  Specification  of  such  a 
structure  would  permit  the  design  of  emergency  decision  training  stra¬ 
tegies  on  a  more  comprehensive,  effective,  and  efficient  basis  than  has 
been  possible  in  the  past.  Similarly,  the  derivation  of  realistic  and 
effective  training  materials,  in  particular  emergency  situation 
scenarios,  would  be  facilitated.  Both  of  these  outcomes  could  be  of 
significant  practical  value  in  addressing  the  aircraft  emergency  problem 
tnrough  improved  training  of  aircrews. 
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Activities  of  Phase  1 


A  mmber  of  activities  were  carried  out  in  the  first  year  of  this 
three-year  program  in  order  to  develop  the  theoretical  base  needed  to 
support  the  succeeding  years'  efforts.  These  activities  included  re¬ 
views  of  SET  and  Boldface  approaches  to  emergency  training;  site  visits 
to  a  number  of  training  squadrons  for  observation,  orientation,  and  In¬ 
terviews  with  flying  personnel;  reviews  of  flight  manuals  and  training 
materials  for  a  number  of  military  aircraft;  an  extensive  period  of 
field  data  collection  and  analysis  of  judgmental  data  derived  from  In¬ 
terviews  with  flying  personnel;  a  review  and  analysis  of  USAF  aircraft 
accident  reports  for  1977;  and  convening  of  a  workshop  of  civilian  and 
military  personnel  to  review  approaches  to  the  analysis  of  aircraft 
emergency  decisions,  aircraft  accident  research,  and  aircrew  emergency 
decision  training. 

Two  major  reports  were  prepared  as  a  result  of  these  activities.  The 
first.  Aircrew  Emergency  Decision  Training:  A  Conference  Report ,  sum¬ 
marizes  the  results  of  the  workshop  held  *n  San  Francisco  in  November 
1978.  The  second  is  this  report  which  presents  the  results  of  the  other 
activities  carried  out  during  Phase  1. 

1.4  Overview  of  Accomplishments 

The  work  of  Phase  1  was  successful  in  providing  a  theoretical  basis  for 
the  design  of  aircrew  emergency  training  programs.  A  taxonomic  struc¬ 
ture  was  derived  which  appears  to  be  of  considerable  value  in  specifying 
the  cognitive  aspects  of  aircraft  emergency  problems.  There  are  two  ob¬ 
vious  applications  of  the  taxonomy  in  training  settings.  The  first  is 
the  specification  of  training  techniques  or  strategies  for  classes  of 
emergency  situations  which  are  related  in  terms  of  cognitive  functions 
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required  for  their  successful  resolution.  The  second  area  of  applica¬ 
tion  is  at  a  more  detailed  level,  namely  in  the  development  of  scenarios 
or  aircraft  emergency  problems  for  use  in  the  training  of  pilots  and 
other  aircrew  members. 

Chapter  7  represents  the  culmination  of  the  year's  work  in  that  the  tax- 
ono<hy  incorporates  both  situational  and  task-specific  elements  as  cogni¬ 
tive  attributes  of  the  decision  tasks  performed  under  emergency  condi¬ 
tions.  There  were  several  steps  preceding  the  development  of  the  taxon¬ 
omy.  The  aggregation  of  situations  which  could  be  considered  within  an 
emergency  training  program  was  reviewed.  A  definition  of  the  emergency 
situation  was  developed  which  limited  the  scope  of  consideration  to  a 
manageable  entity--known  malfunctions.  Representational  models  of  the 
objective  (external)  emergency  situation,  decision  processes,  and  cogni¬ 
tive  functioning  were  proposed  as  a  way  of  characterizing  the  situation¬ 
al  and  behavioral  aspects  of  an  emergency  malfunction.  The  taxcnomic 
structure  was  then  derived  after  consideration  of  the  cognitive  elements 
of  the  three  representational  models. 

On  the  basis  of  the  taxonomy,  three  classes  of  emergency  situations  were 
found  to  be  of  interest:  Situation  1  (mostly  predictable).  Situation  2 
(partly  predictable),  and  Situation  3  (unpredictable).  Initial  training 
guidelines  are  suggested  in  light  of  the  cognitive  requirements  of  each 
class. 

The  taxonomy  also  provides  a  framework  for  emergency  scenario  genera¬ 
tion.  Situational  and  behavioral  aspects  of  emergencies  are  covered  at 
a  level  of  detail  which  allows  systematic  identification  of  their  cogni¬ 
tive  elements.  Thus,  the  utility  of  the  taxonomy  in  specifying  scenario 
components  lies  in  the  ability  to  correlate  various  and  seemingly 
disparate  elements  of  a  given  problem  (or  set  of  problems)  in  cognitive 
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terms.  This  provides  the  capability  to  manipulate  scenario  parameters 
in  a  systematic  fashion  so  as  to  ensure  that  training  experiences  are 
managed  (and  evaluated)  in  terms  of  a  comprehensive  and  unifying 
factor--cognitive  functions. 

The  material  of  Chapter  6  is  another  key  product  of  the  present  study, 
that  is,  the  preliminary  classification  of  emergency  situations  accord¬ 
ing  to  the  performance  requirements  of  these  situations  as  dictated 
largely  by  the  physical  nature  of  high-performance  aircraft  operations. 
This  material  supports  the  work  of  Chapter  7  since  it  facilitates  the 
identification  of  emergency  situations  which  are  candidates  for  special 
emphasis  in  decision  training  programs.  The  classification,  which 
derives  from  consideration  of  the  risk,  time  pressures,  and  complexity 
of  decision  making  tasks  associated  with  specified  malfunctions,  lends 
an  objective  frame  of  reference  to  the  theoretical  tools  provided  by  the 
representational  models  and  taxonomy  of  Chapter  7. 

Taken  together,  the  schema  and  data  of  Chapter  6  and  the  taxonomic 
structure  of  Chapter  7  appear  to  provide  a  means  to  address  training 
program  design  both  in  terms  of  the  objective  (physical)  and  cognitive 
factors  of  aircraft  emergencies.  Work  in  Phase  2  will  be  aimed  at  re¬ 
fining  and  extending  the  tools  developed  as  a  result  of  Phase  1  activi¬ 
ties. 


1.5  Report  Contents 

Chapters  2  through  7  of  this  report  describe  the  results  of  six  distinct 
phases  of  effort  carried  out  during  the  first  year  of  activity.  Chapter 
i  presents  a  review  of  reliant  background  literature,  which  includes 
decision  making,  decision  making  as  a  flying  skill,  limitations  on  the 
decision  maker,  decision  training,  and  the  effects  of  stress.  Chapter  3 
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briefly  reviews  the  events  and  resulting  recommendations  of  the  Aircrew 
Emergency  Decision  Training  Workshop. 

Chapter  4  reviews  current  approaches  to  aircrew  emergency  decision 
training  and  relates  them  to  relevant  psychological  theories  of  learning 
and  cognitive  activity.  Chapter  5  contains  the  results  of  an  analysis 
of  ib 7  USAf  aircraft  accidents  which  occurred  in  1977  as  well  as  a  dis¬ 
cussion  of  the  value  of  such  analyses  in  designing  training  programs  and 
training  materials.  Chapter  6  presents  a  preliminary  classification  of 
aircraft  emergency  situations  in  terms  of  several  situational  and  deci¬ 
sion  making  attributes.  The  classif ication  is  based  on  data  derived 
from  interviews  with  experienced  military  flying  personnel.  Chapter  7 
represents  the  major  theoretical  outcome  of  the  first  year’s  efforts. 
Three  representational  models  are  proposed  for  the  emergency  situation 
and  a  taxonomic  structure,  trfiich  treats  decision  aspects  of  emergency 
situations,  is  derived.  Cognitive  theory  is  used  to  link  variables  of 
emergency  situations,  as  identified  in  the  taxonomy,  to  appropriate 
training  methods. 
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2.  BACKGROUND  LITERATURE 


2.1  Introduction 


As  an  Initial  step  in  the  work  of  Year  1,  a  review  of  the  relevant 
literature  was  undertaken  to  provide  a  background  for  the  research  and 
development  efforts  to  be  carried  out.  A  computerized  search  was  car¬ 
ried  out,  relying  principally  on  NTIS  and  on  Psychological  Abstracts. 
Abstracts  were  selected  and  reviewed  for  a  number  of  articles,  books, 
and  technical  publications  which  seemed  relevant  to  the  present  work. 
Topics  of  interest  Included  decision  making,  emergency  procedures,  air¬ 
crew  training  and  performance,  decision  training,  and  performance  under 
stress.  While  the  literature  on  each  of  these  separate  topics  Is  large, 
and  in  some  cases  voluninous,  aircrew  emergency  decision  training,  as  a 
combined  topic,  has  received  relatively  little  attention. 

The  results  of  this  review  are  summarized  in  this  chapter.  Certain  re¬ 
view  articles  proved  to  be  particularly  relevant  and  comprehensive. 

These  include  works  by  Goodman,  Fischhoff,  Lichtenstein  and  Slovic, 

197b,  Kanarick,  1969;  Nickerson  and  Feehrer,  1975;  Prophet,  19/6;  Slo¬ 
vic,  1976;  and  Vaughan  and  Havor,  1972.  The  topics  covered  below  in¬ 
clude  decision  making,  decision  making  as  a  flying  skill,  limitations  on 
decision  making,  decision  training,  and  the  effects  of  stress. 

2.2  Decision  Making 

Decision  science  has  become  an  area  of  growing  interest  to  defense, 
business,  medical  and  other  organizations.  This  interest  has  evolved 
from  the  need  to  improve  the  quality  of  decisions  by  ensuring  that  ac¬ 
tion  alternatives  are  chosen  which  maximize  the  expected  gain  to  be 
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derived  by  the  individual  or  organization.  Nickerson  and  Feehrer  (1975) 
point  out,  however,  that: 


Much  has  been  written  about  the  importance  of  decision 
making  for  industry,  for  government,  for  the  military 
and  for  rational --or  at  least  reasonable- -people  in 
general.  Moreover,  a  great  deal  of  research  has  been 
conducted  on  decision-making  behavior.  In  spite  of 
these  facts— or  perhaps  because  of  them—there  is  not 
general  agreement  concerning  what  decision  making  is, 
how  it  should  be  done,  how  it  is  done,  how  to  tell 
whether  it  is  done  well  or  poorly,  and  how  to  train 
people  to  do  it  better,  (p.  1) 


However,  when  the  term  "decision  making"  is  used  in  commercial  and  mili¬ 
tary  contexts,  there  is  informal  agreement  concerning  the  components  of 
decision  situations,  which  include  "fairly  well-defined  objectives,  sig¬ 
nificant  action  alternatives,  relatively  high  stakes,  inconclusive  in¬ 
formation  and  limited  time  for  decision."  (Nickerson  and  Feehrer,  p.  1) 

There  are  many  ways  to  classify  tfe  various  tasks  that  the  decision  mak¬ 
er  may  be  required  to  perform.  Nickerson  and  Feehrer  (1975)  found  that 
the  most  satisfactory  scheme  recognizes  eight  aspects  of  decision  mak¬ 
ing:  information  gathering,  data  evaluation,  hypothesis  generation, 
problem  structuring,  hypothesis  evaluation,  preference  specification, 
action  selection,  and  decision  evaluation.  Not  all  these  tasks  are  in¬ 
volved  in  every  decision,  nor  are  they  all  equally  difficult.  For  exam¬ 
ple,  even  though  alternative  selection  is  central  to  decision  making, 
the  problem  of  choosing  among  possible  courses  of  action  Is  frequently 
far  simpler  than  that  of  discovering  what  one's  options  are  in  the  first 
place,  or  of  being  consistent  in  assigning  preferences  to  possible  deci¬ 
sion  outcomes. 
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Each  of  these  eight  aspects  of  the  decision  situation  represents  a  prob¬ 
lem  necessitating  some  decision  making.  Frequently,  several  preliminary 
problems  may  have  to  be  solved  before  even  considering  the  decision 
which  is  of  primary  concern.  Such  decisions  include  the  acquisition  of 
Information  necessary  to  set  the  stage  for  the  primary  decision.  The 
problems  of  deciding  how  much  time  is  available  for  buying  additional 
information  and  what  are  reasonable  costs  for  it,  must  also  be  con¬ 
sidered. 

The  linkage  of  decision  making  and  problem  solving  is  also  found  in 
Dieterly's  (1978)  work  on  the  clarification  process  model,  which  was 
developed  in  part  as  a  building  block  for  training  programs  in  decision 
making  for  managers  of  aircrew  and  aircraft  systems.  Oieterly  ties  to¬ 
gether  concepts  in  decision  making  and  problem  solving  through  their 
underlying  reliance  on  information  flow  and  information  processing.  His 
model  reflects  the  increasing  interest  in  a  unified  psychological  ap¬ 
proach  to  decision  making,  an  interest  which  parallels  the  recent  in¬ 
terest  in  and  growth  of  the  cognitive  movement  in  psychology. 

A  second  noteworthy  aspect  of  Nickerson  and  Feehrer's  characterization 
of  the  tasks  performed  by  the  decision  maker  is  the  attempt  to  identify 
and  define,  to  the  extent  possible,  e.ght  tasks  (or  sub-processes)  per¬ 
formed  by  decision  makers.  Although  a  number  of  taxonomic  systems  for 
the  description  of  decision  activities  have  been  proposed,  the  essential 
point  is  that  a  variety  of  investigators  have  found  it  useful  to  attempt 
to  subdivide  "decision  making"  into  specific  activities  or  sub¬ 
processes,  each  of  which  may  separately  be  more  amenable  to  investiga¬ 
tion  than  decision  making  taken  as  a  whole.  This  approach  is  compatible 
with  the  instructional  systems  development  (ISO)  approach  to  training, 
vrfiich  relies  heavily  on  detailed  analysis  of  tasks  to  be  performed  and 
development  of  instructional  materials  designed  to  facilitate  learning 
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of  the  knowledge  and  skills  required  for  successful  task  performance. 

Few  would  seriously  argue  that  a  rigid  and  overly  simplified  ISO  ap¬ 
proach  to  decision  training  is  either  useful  or  an  immediate  possibili¬ 
ty.  Many  would  question  whether  decision  skills  could  ever  be  developed 
by  relying  on  such  "reductionist"  training  methods.  Nevertheless,  the 
trend  toward  isolation  and  description  of  decision  sub-processes  will 
encourage  and  support  more  systematic  attempts  to  develop  decision 
training  programs  that  are  more  systematic  than  those  realized  in  the 
past. 


2.3  Decision  Making  as  a  Flying  Skill 

An  extensive  survey  of  the  behavioral  science  literature  dealing  with 
the  subject  of  flying  skills,  in  particular  the  long-term  retention  of 
such  skills,  was  carried  out  by  Prophet  (1976).  He  characterized  the 
nature  of  the  military  pilot's  task  as  follows: 


It  is  clear  that  the  tasks  the  pilot  of  a  modern 
military  aircraft  must  perform  are  many  and  complex. 
There  are  few  task  situations  that  demand  as  much 
of  the  performer  in  terms  of  physical  strength  and 
endurance,  fine  perceptual  and  motor  discriminations, 
cognitive  functioning,  verbal  communication  skills, 
decision  making,  and  the  like,  as  does  that  of 
flying  an  aircraft,  (p.  14) 


According  to  Prophet,  the  critical  aspects  of  advanced  flying  skills  are 
primarily  cognitive,  dealing  with  identification  and  acquisition  of 
relevant  information  (in  terms  of  both  tactical  and  aircraft  situa¬ 
tions),  the  processing  of  such  information,  decision  making,  system 
management  (including  tactical,  aircraft,  and  human  systems),  and  simi¬ 
lar  “higher  level"  functions. 
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Nearly  two  decades  earlier,  Williams  and  Hopkins  (1958)  had  expressed 
similar  views.  They  felt  that  cognitive  activity,  and  decision  making 
in  particular,  was  becoming  an  Increasingly  important  aspect  of  operat¬ 
ing  military  aircraft: 


A  reasonable  extrapolation  of  the  past  trend  in 
operator  tasks  to  the  future  suggests  that  operators 
will  become  less  and  less  concerned  with  continuous 
manual  control  tasks  and  more  and  more  concerned 
with  the  interpretation  of  information  assembled  from 
a  variety  of  sources  and  displayed  ’‘artificially" 
within  the  cockpit  and  with  the  choice  of  operating 
mode  based  upon  the  Information  received.  This  kind 
of  activity  corresponds  closely  with  what  is 
commonly  known  as  the  exercise  of  judgment  or  the 
making  of  decisions,  (p.  3) 


Although  their  analysis  of  pilot  functions  in  the  F-1Q6  led  Williams  and 
Hopkins  to  conclude  that  decision  making  was  an  increasingly  important 
function,  they  viewed  decision  making  in  a  rather  narrow  vein,  suggest¬ 
ing  that  the  courses  of  action  open  to  the  pilot  are  built  into  the  sys¬ 
tem.  According  to  Williams  and  Hopkins,  the  pilot's  decision  functions 
are  concerned  with  the  diagnosis  of  the  state  of  the  system  and  only 
rarely  with  the  cho.o  of  a  course  of  action  to  pursue,  because  the  mis¬ 
sion  is  carefully  planned  in  advance,  with  modes  of  operation  provided 
for  each  major  state  in  which  it  Is  expected  the  system  will  find  it¬ 
self.  The  pilot's  decision  is  seen  as  a  diagnosis--a  detection  and 
recognition  of  the  state  of  the  system--and,  having  done  this,  he  ad¬ 
justs  the  equipment  to  operate  in  the  mode  specified  in  advance. 

Pilot  decision  making,  according  to  this  view,  is  limited  to  situation 
recognition  and  is  followed  by  execution  of  a  wel 1- rehearsed,  pre¬ 
planned  response  sequence.  Training  for  decision  making,  in  this  case. 
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would  focus  on  learning  to  screen  and  rapidly  classify  situations  into 
predetermined  categories  (template  matching)  and  to  associate  a  standard 
response  with  each  category.  Situation  recognition  and  rapid  response 
execution  are  undoubtedly  important  elements  of  decision  making  and  fly¬ 
ing  skill.  Other  investigators,  however,  have  taken  a  broader  view  of 
the  decision  requirements  placed  on  the  pilot.  With  respect  to  the  em¬ 
ergency  situation,  which  is  perhaps  the  ultimate  test  of  aircrew  deci¬ 
sion  skills,  Thorpe,  Martin,  Edwards,  and  Eddowes  (1976)  provide  a  de¬ 
tailed  picture  of  the  complex  decision-making  activities  involved: 


During  the  course  of  any  emergency  situation.. .the  pilot 
should:  (a)  maintain  aircraft  control,  (b)  analyze  the 
situation  and  take  the  proper  action,  and  (c)  land  as 
soon  as  practicable. 

Now  consider  the  likely  course  of  events:  the  pilot  is 
somewhere  along  in  the  mission,  attending  to  the 
mission  requirements,  and  unexpectedly  an  emergency 
occurs.  The  emergency  may  be  indicated  by  warning  lights, 
an  abnormality  in  instrument  readings,  abnormal  flight 
control  responses,  strange  noises,  vibrations,  or  any 
combination  of  a  number  of  these  or  other  cues.  Some 
of  these  cues  are  easily  detected,  others  are  more 
subtle  and  may  not  be  immediately  perceived.  Once  the 
pilot  detects  the  cues,  he  must  do  two  things  simul¬ 
taneously:  continue  to  fly  the  aircraft,  and  analyze 
the  situation.  Accomplishing  these  in  a  multicrew  air¬ 
craft  may  not  be  as  taxing  as  in  a  single  place  aircraft, 
provided  crew  coordination  does  not  break  down.  But  in 
a  single  place  aircraft  under  some  conditions,  main¬ 
taining  aircraft  control  alone  will  be  a  demanding 
task,  likewise,  analyzing  the  situation  may  be  a 
simple  diagnostic  process  or  it  could  be  considerably 
more  complex,  involving  complicated  information 
seeking.  The  appropriate  response  could  be  a  simple 
response  sequence,  or  it  could  be  an  extended 
sequence  of  inputs. 

After  recognizing  and  analyzing  the  emergency,  irtiile 
maintaining  aircraft  control,  the  pilot  must  determine 


the  consequences  of  various  responses  on  the  rest  of 
the  mission.  Usually  this  will  require  a  plan  of 
recovery.  The  pilot  must  anticipate  the  interaction 
of  his  corrective  actions  with  the  immediate  problem 
solution  and  with  the  safe  landing  or  conclusion  of 
the  mission.  Thus,  he  must  know  where  he  Is,  where 
he  is  going,  and  how  he  is  going  to  land  safely 
when  he  gets  there.  Failure  to  think  through  these 
phases  of  the  recovery  can  compound  the  emergency. 

The  fundamental  cognitive  activites  of  the  pilot 
during  the  emergency  are  the  detection  of  the  cues  or 
symptoms  which  signal  the  onset  of  the  emergency,  the 
diagnostic  determination  of  what  is  wrong,  the 
decision  making  processes  which  consider  viable 
alternative  courses  of  action,  the  selection  of  the 
most  suitable  response,  and  the  execution  of  that 
response.  The  need  for  good  judgment  during  these 
activities  is  obvious,  (pp.  7-8) 


Thorpe  et  al.  question  whether  standard  emergency  training  procedures, 
which  often  emphasize  learning  of  predetermined  procedural  responses, 
satisfactorily  address  the  development  of  the  decision  skills  needed  In 
emergencies.  If  current  “training  discourages  judgment  or  makes  it 
harder  to  exercise,  it  follows  that  an  alternative  training  approach 
should  be  considered.  Is  It  possible  to  train  good  judgment  as  well  as 
procedural  accuracy?-  (pp.  7-8). 

Similarly,  Prophet  (1976)  points  out  that  while  the  training  of  basic 
flying  skills  is  reasonably  well  understood,  less  is  known  about  train¬ 
ing  more  advanced  skills  such  as  decision  making  because  little  research 
has  been  done  on  the  nature,  development,  maintenance,  and  retraining  of 
the  higher  level  flight  skills  characteristic  of  the  professional  USAF 
pilot.  Prophet  lists  a  number  of  areas  such  as  changes  In  ability/skill 
with  time  and  experience.  Information  processing  concepts,  multi-task 
residual  attention  capabil Ities,  and  learning  and  performatory  stra- 
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tegies  for  higher  skill  levels,  which  would  be  profitable  to  Investi¬ 
gate.  He  identifies  the  need  for  a  better  understanding  of  the  factors 
involved  in  the  acquisition,  maintenance,  and  retraining  of  higher  level 
pilot  skills.  There  may  or  may  not  be  fundamental  differences  in  the 
principles  underlying  effective  training  for  basic  and  higher  skills. 
Such  differences,  however,  can  only  be  discovered  by  first  defining  the 
nature  of  these  higher  skills  and  establishing  specific  objectives  for 
their  training. 

One  particular  area  of  concern  is  the  need  for  advancing  the  technology 
of  design  and  use  of  simulation  for  the  training  of  higher  level  pilot 
skills.  Training  devices  vary  from  quite  simple  devices  to  complex 
flight  and  weapons  systems  simulators.  While  various  training  devices 
can  be  used  with  high  cost  effectiveness  in  flight  skills  maintenance 
and  retraining  programs,  very  little  is  known  concerning  their  effec¬ 
tiveness  in  training  higher  order  flight  skills.  Clearly,  this  is  an 
important  area  for  further  research. 

2.4  Limitations  on  Decision  Making 

The  study  of  human  decision  making  behavior  reveals  a  number  of  defi¬ 
ciencies  which  accompany  the  different  component  tasks  which  constitute 
decision  making  (Hammel 1  and  Kara,  1970).  In  the  literature,  the  term 
"deficiencies"  is  used  in  two  ways:  (1)  to  refer  to  stereotyped  ways  of 
behaving  suboptimal ly,  such  as  the  tendency  of  humans  to  be  overly  con¬ 
servative  in  their  application  of  probabilistic  information  to  the 
evaluation  of  hypotheses,  and  (2)  to  refer  to  basic  human  cognitive  lim¬ 
itations  of  memory,  attention  span,  and  information  processing,  which 
prevent  most  people  from  weighing  more  than  a  small  number  of  factors  in 
arriving  at  a  preference  among  alternatives  without  procedural  help. 
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The  following  provides  an  overview  of  man's  characteristic  performance 
in  those  empirical  tasks  that  have  been  studied  as  components  of  deci¬ 
sion  making  behavior.  Three  main  categories--problem  recognition,  si¬ 
tuation  diagnosis,  and  action  selection-will  be  examined  for  this  pur¬ 
pose.  Much  of  the  material  presented  is  based  on  the  excellent  summary 
prepared  by  Vaughan  and  Mavor  (1972). 

2.4.1  Problem  Recognition.  This  aspect  of  the  decision  process  basi¬ 
cally  Involves  the  monitoring  of  an  ongoing  action  in  terms  of  its  im¬ 
pact  on  a  given  situation,  comparing  key  aspects  or  dimensions  of  the 
situation  to  acceptable  limits,  and  determining  whether  the  action  is 
still  appropriate  to  the  situation.  Options  available  to  the  decision 
maker,  once  a  problem  is  identified,  are  to  initiate  a  new  action,  to 
modify  or  terminate  an  ongoing  action,  or  to  continue  present  actions. 

Available  empirical  evidence  suggests  that  men  tend  to  err  on  the  side 
of  conservatism  in  this  task.  For  example,  Vaughan,  Virnelson,  and 
Franklin  (1964)  asked  experienced  army  officers  to  monitor  a  series  of 
messages  that  Indicated  the  need  to  change  the  axis  of  advance  in  a 
simulated  attack  scenario.  With  only  one  exception,  officers  did  not 
modify  the  ongoing  action  plan,  nor  did  they  anticipate  the  possibility 
of  changing  the  plan,  in  spite  of  a  series  of  messages  indicating  this 
need  with  Increasing  urgency. 

2.4.2  Situation  Diagnosis.  Man  is  a  weak  diagnostician.  Summarizing 
results  from  several  studies  of  clinical  diagnosis,  Goldberg  (1968)  con¬ 
cluded  that  diagnostic  judgments  are: 


(1)  Unreliable  over  time. 


(2)  Unreliable  across  diagnosticians. 

(3)  Only  marginally  related  either  to  experience  of  the  man  or 
to  his  confidence  in  the  accuracy  of  his  judgments. 

(4)  Only  slightly  affected  by  the  amount  of  available  informa¬ 
tion. 

(5)  Generally  of  low  validity. 

These  discouraging  results  can  be  traced  to  the  complexity  of  the  diag¬ 
nostic  task,  particularly  where  configurational,  non-linear  cue  patterns 
are  a  component  of  the  problem.  Diagnosis--and  military  diagnosis  is 
not  an  exception--is  primarily  a  task  area  requiring  a  cycling  of  induc¬ 
tive  inference  processes  that  build  diagnostic  categories  from  items  of 
data  and  their  interactions,  and  deductive  processes,  for  testing  a 
given  diagnostic  category  against  available  data.  Experts  in  clinical 
psychology,  medicine,  psychiatry,  military  intelligence,  and  the  like, 
typically  view  their  work  as  involving  complex  interpretations  of  confi- 
gural  cue  patterns.  However,  carefully  planned  studies  of  the  process, 
using  qualified  diagnosticians  as  subjects,  have  not  revealed  much  use 
of  these  configural  cues  in  the  outcomes  of  their  judgments.  Simple, 
li  lar,  additive  models  typically  account  for  more  than  90%  of  the  out¬ 
comes  of  clinical  diagnosis. 

Edwards  (1963)  presented  evidence  from  non-clinical  studies  that  man  is 
a  relatively  good  probability  estimator  for  single  items,  but  poor  at 
aggregating  a  number  or  probability  estimates  to  form  a  conclusion.  Ad¬ 
ditional  evidence  and  discussion  of  this  misaggregation  effect  were  pro¬ 
vided  by  Slovic  and  Lichtenstein  (1971)  and  Rapoport  and  Wallsten 
(1972).  The  conclusion  that  men  do  not  do  well  at  extracting  informa- 


2-10 


1 1  on  from  available  data  is  supported  by  the  findings  of  Vaughan,  Frank¬ 
lin,  and  Johnson  (196b);  namely,  that  ambiguous,  partial,  and  conflict¬ 
ing  information  items  are  ignored  as  inputs  to  the  planning  process. 
Vaughan  et  al .  had  a  group  of  experienced  arny  officers  study  a  series 
of  map  problems  that  contained  partial  and  ambiguous  information  about 
enemy  strength  and  disposition.  Schemes  of  maneuver  w®re  planned  on  the 
basis  of  Information  categories  that  were  known  to  be  reliable:  the  mis¬ 
sion  order,  the  terrain,  and  the  available  resources.  It  was  found  that 
information  about  the  enemy  was  not  accorded  various  possible  interpre¬ 
tations,  and  did  not  influence  the  planning  process. 

2.4.3  Action  Selection.  The  process  of  selecting  an  action  (or  a 
complex  of  serial  or  contingent  actions)  assumes  the  existence  of  a  di¬ 
agnostic  category  or  set  of  categories  that  define  the  state(s)  for 
which  an  action  response  or  plan  is  required.  Selecting  an  action  (or 
set  of  actions)  involves  the  following  subtasks: 

(1)  Formulation  of  alternative  action  possibilities. 

(2)  Formulation  of  appropriate  criteria  for  assessing  alterna¬ 
tives. 

(3)  Assignment  of  differential  weights  or  priorities  to  the 
criteria. 

(4)  Assessment  of  alternatives  against  the  criteria. 

Ideally,  this  set  of  subtasks  is  to  be  performed  iteratively  at  succes¬ 
sive  levels  of  detail.  A  variety  of  prescriptive  models  of  decision 
making  of  this  kind  exist  in  the  military.  Empirical  studies  of  persons 
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responsible  for  decision  making  of  this  sort  in  actual  environments  are 
summarized  below  according  to  the  main  subtasks. 

Formulation  of  Action  AUernati ves  and  Evaluation  Criteria.  When  a 
pre-establ ished  set  of  action  alternatives  does  not  exist,  man  is  re¬ 
quired  to  develop  alternatives  from  a  set  of  resources  which  he  can  draw 
upon  or  use  in  one  of  several  ways.  In  complex  situations,  man  ap¬ 
parently  has  difficulties  in  creating  alternatives  {Gagliardi,  Hussey, 
Kaplan,  and  Matteis,  1965;  Vaughan  et  al.,  1966).  Formulation  of 
evaluation  criteria  appears  to  present  comparable  difficulties  (e.g., 
Schroder,  1965).  Moreover,  there  is  evidence  that  criterion  identifica¬ 
tion  is  correlated  with  action  formulation  such  that  a  decision  maker 
will  tend  to  only  consider  criteria  that  support  the  action  alternative 
he  has  created. 

Assignment  of  Differential  Weights  to  Criteria.  The  1 imited  evidence 
available  suggests  that  experienced  decision  makers  and  problem  solvers 
are  excellent  criteria  evaluators.  Vaughan  et  al.  ( 1 964 ) ,  for  example, 
asked  experienced  submarine  commanders  and  officers  to  assign  quantita¬ 
tive  weights  to  seven  criteria  affecting  he  desirability  of  a  running 
depth  for  two  tactical  situations.  Criterion  weights  were  highly  reli¬ 
able  over  time,  consistent  within  subjects,  and  differentiated  appropri¬ 
ately  between  tactical  situations. 

Assessment  of  Alternatives  Against  Criteria.  Simultaneous  consideration 
of  multiple  alternatives  portrayed  against  multiple  criteria  in  a  deci¬ 
sion  matrix  quickly  becomes  too  complex  for  easy  resolution.  For  exam¬ 
ple,  Hayes  (19b<0  found  decreases  in  decision  quality  and  increases  in 
time  required  as  criteria  were  increased  from  two  to  eight  for  four- 
alternative  and  for  eight-alternative  decision  problems.  Also,  Connolly 
and  his  associates  conducted  a  series  of  experiments  at  Hanscomb  Field 


to  assess  the  appropriateness  of  weapon  selection  decisions  by  experi¬ 
enced  Air  Force  officers  in  a  simulated  air  defense  environment  (Connol¬ 
ly,  Fox,  and  McGoldrick,  1961;  Connolly,  McGoldrick  and  Fox,  1961;  Fox 
and  Vance,  1961).  Although  instructed  to  use  three  criteria  in  the 
selection  of  weapons  to  targets  (minimize  damage  to  defended  area,  des¬ 
troy  maximum  number  of  threatening  objects,  and  conserve  counter¬ 
weapons),  actual  selections  reflected  a  disproportionate  weighting  of 
the  three  factors. 

2.4.4  Summary.  The  fi  lowing  picture  emerges  of  people’s  performance 
in  making  complex  decisions  (Vaughan  and  Mavor,  1972): 

(1)  Humans  are  slow  to  initiate  action  and  conservative  in 
their  estimates  of  highly  probable  situations. 

(2)  When  humans  act  or  accept  a  diagnosis,  they  are  reluctant 
to  change  an  established  plan  or  a  situational  estimate 
when  the  available  data  indicate  that  they  should. 

(3)  They  are  generally  poor  diagnosticians. 

(4)  Humans  are  not  particularly  inventive  and  tend  to  adopt  the 
first  solution  developed. 

(51  They  find  it  difficult  to  use  more  than  one  or  two  criteria 
at  a  time  in  evaluating  actions  and  tend  to  identify  cri¬ 
teria  that  reflect  favorably  on  the  action  being  developed. 

(6)  Humans  tend  to  use  only  concrete,  high-confidence  facts  in 
planning  and  prefer  to  Ignore  or  reduce  the  importance  of 
ambiguous  or  partial  data. 


(7)  They  are  good  judges  of  the  probability  of  single  items  of 
information,  given  alternative  hypotheses. 

(8)  They  are  very  go.xl  judges  of  the  relative  Importance  of 
those  criteria  that  can  be  identified. 

Different  approaches  to  correct  deficiencies  in  human  decision  making 
have  been  suggested  (Schrenk,  1969).  There  are  basically  three  ways  to 
improve  human  decision  performance: 

(1)  Selection:  Insure  that  decisions  are  made  only  by  Indivi¬ 
duals  who  are  competent  to  make  them. 

(2)  Decision  Aiding:  provide  decision  makers  with  procedural 
and  technical  aids  to  compensate  for  their  own  limitations. 

(3)  Decision  Training:  attempt  to  improve  the  decision-related 
skills  of  people  in  decision-making  positions. 

Decision  aiding  and  decision  training  can  be  viewed  as  compl ament ary  to 
one  another.  While  decision  training  attempts  to  improve  decision  mak¬ 
ing  behavior  by  training  out  deficiencies  and  highlighting  limitations, 
decision  aiding  provides  the  decision  maker  with  procedural  and  techni¬ 
cal  aids  which  let  him  go  beyond  his  own  limitations  in  the  process  of 
decision  making.  Much  research  work  has  been  performed  on  decision  aid¬ 
ing,  »rfiile  decision  training  has  been  rarely  investigated. 

2.5  Decision  Training 


2.5.1  Current  Programs.  There  are  a  number  of  decision  training  pro¬ 
grams  that  are  currently  being  implemented  with  some  success.  These 


programs  are  either  task  specific  or  treat  only  limited  aspects  of  the 
decision  making  process.  Some  of  the  programs  are  used  In  operational 
contexts. 

Einhorn  and  Hogarth  (1975)  have  developed  an  approach  to  teaching 
multi-attribute  utility  analysis  to  top-level  executives  and  middle- 
level  managers.  While  their  system,  termed  “An  Idiot's  Guide  to  Deci¬ 
sion  Making,"  maintains  a  reasonable  degree  of  independence  with  respect 
to  any  specific  domain,  it  covers  only  one  method  of  alternative  evalua¬ 
tion.  Other  tasks  involved  in  the  decision  making  process,  as  well  as 
the  interrelationships  among  such  tasks,  are  ignored. 

Decisions  and  Designs,  Inc.  has  developed  a  decision-aiding  system 
called  Rapid  Screening  of  Options.  The  system  involves  an  interactive 
computer  program  that  simplifies  a  decision  analysis  by  focusing  on  a 
limited  number  of  alternatives  and  on  the  major  causes  of  uncertainty 
(Selvidge,  1976).  The  training  aspect  of  the  system  consists  of 
displaying  the  expected  value  associated  with  each  alternative  evalua¬ 
tion  and  does  not  cover  other  elements  of  decision  making  such  as  prob¬ 
lem  recognition,  alternative  development,  and  the  optimal  sequencing  and 
effort  allocation  for  these  subprocesses. 

Hanroond,  Stewart,  Brehmer,  and  Steinman  (1975)  present  judgment  as  the 
key  element  of  a  decision  making  process.  They  assume  that  if  people 
are  taught  the  theory  behind  judgment  analysis  and  are  then  trained  in 
increasingly  difficult  applications  of  task  situations,  they  will  even¬ 
tually  be  able  to  analyze  any  problem  properly.  Based  on  this  assump¬ 
tion,  their  training  system  focuses  strongly  on  judgmental  aspects  and 
ignores  the  other  elements  of  the  decision  making  process. 
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The  Decision  Analysis  Group  at  Stanford  Research  Institute  conducts 
several  different  training  programs  for  decision  makers.  Through  these 
programs,  trainees  are  expected  to  learn  that  decision  theoretic  metho¬ 
dology  exists,  that  uncertainties  and  utilities  can  be  quantitatively 
estimated,  and  that  they  can  begin  to  structure  and  work  through  their 
own  decision  problems.  While  the  programs  are  enriched  by  a  reasonable 
degree  of  generality  and  completeness,  they  do  not  provide  the  required 
link  between  decision  training  and  specific  application  areas. 

Michigan  State  University's  Medical  School  approach  to  the  training  of 
physicians  is  based  on  a  total  curriculun  design  wherein  decision 
analysis  is  integrated  within  specific  content  areas  (Allal,  1973;  El- 
stein,  Shulman,  and  Sprafka,  1978).  It  assumes  that  the  diagnostic 
phase  of  medicine  consists  of  generating  hypotheses  about  what  the  medi¬ 
cal  problem  might  be,  distinguishing  the  relevant  from  the  irrelevant 
features  of  the  case,  and  then  systematically  gathering  information  to 
test  and  compare  alternative  hypotheses.  The  program  is  task-specific 
and  relies  heavily  on  the  case-study  method. 

The  Los  Angeles  Police  Academy's  “shoot/no  shoot"  training  is  an  example 
of  a  task-specific  decision  training  program.  Although  the  program  cov¬ 
ers  no  formal  training  in  either  probability  theory  or  decision  theory, 
it  includes  extensive  courses  in  the  established  important  attributes 
that  should  be  considered  when  deciding  whether  or  not  to  shoot  in  a 
given  situation.  There  are  no  relative  weighting  schemes  for  the  attri¬ 
butes  nor  decision  rules  that  translate  the  utilities  of  the  attributes 
imo  a  decision.  There  are,  however,  general  guidelines  that  help  the 
cadets  make  the  decision.  Because  of  time  criticality,  the  cadets  are 
taught  to  prune  the  decision  tree  before  the  actual  decision  situation 
arises.  The  training  system  Is  tailored  for  the  specific  task  involved 
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and  lacks  ^e  generality  and  completeness  desired  in  a  decision  training 
program. 

The  Kepner-Tregoe  process  is  an  explicit,  rational  system  for  gathering 
and  formatting  data  for  decision  making  and  problem  solving  (Kepner  and 
Tregoe,  1965).  It  is  taught  through  in-depth  workshops  which  intermin¬ 
gle  lectures  and  a  graded  series  of  exercises.  The  process  has  four 
main  components:  (1)  situation  appraisal,  (2)  problem  analysis,  (3)  de¬ 
cision  analysis,  and  (4)  potential  problem  analysis.  Proponents  of  the 
process  claim  as  advantages  that  it  results  in  a  visible  (traceable) 
process,  is  streamlined  and  efficient,  forces  actions  and  responses,  and 
helps  the  decision  maker  stay  on  the  subject.  Furthermore,  the  require¬ 
ment  to  explicitly  write  out  the  steps  in  the  analysis  of  each  problem 
is  felt  to  result  in  a  more  conspicuous  identification  of  assumptions 
and  biases  than  is  usually  the  case  in  decision  analysis.  Little  data 
exist  with  which  to  evaluate  the  Kepner-Tregoe  process,  but  Its  contin¬ 
ued  commercial  success  suggests  that  it  is  of  considerable  value  in  the 
training  of  new  managerial  personnel. 

Perceptronics,  Inc.  has  recently  developed  a  decision  aiding  system 
specifically  Intended  to  facilitate  group  decisions  (Leal  and  Pearl, 
1977).  The  system  is  designed  to  compensate  for  some  of  the  deficien¬ 
cies  of  hunans  with  respect  to  handling  large  amounts  of  data  and  per¬ 
forming  complex  calculations.  While  the  decision  aiding  system  is  not  a 
training  device  as  such,  it  allows  groups  charged  with  decision  making 
responsibilities  to  focus  on  problem  exploration  and  value  clarifica¬ 
tion.  Through  repeated  sessions  with  the  system,  it  is  likely  that  de¬ 
cision  groups  will  develop  more  efficient  and  focussed  techniques  for 
problem  definition  and  development  of  consensus.  The  system  involves 
Interactive  elicitation  of  decision  trees.  Including  on-line  sensitivity 
analysis,  and  multi-attribute  analysis  of  group  utility  values  at  criti- 
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cal  points  in  the  tree.  An  interactive  computer  syscem  processes  group 
member  responses  which  are  entered  through  a  set  of  response  devices. 

On  a  large  screen,  the  system  displays  decision  trees,  event  nodes,  and 
the  range  of  group  members'  utilities  for  various  outcomes.  Initial 
realization  of  the  system  has  been  in  the  form  of  a  group  decision  room 
which  is  an  instrumented  conference  room.  A  trained  staff  member, 
termed  an  "intermediator,"  manages  the  general  group  process  and  assists 
with  data  entry  and  selection  of  computer  displays.  Demonstration  stu¬ 
dies  with  the  system  indicate  that  it  has  considerable  potential  for  im¬ 
proving  the  quality  of  group  decisions  as  well  as  for  reducing  the  time 
taken  to  arrive  at  a  decision. 

2.5.2  Recommendations.  Goodnan  et  al .  (1976),  in  the  report  of  a  re¬ 
cent  conference  on  the  training  of  decision  makers,  identify  some  prior¬ 
ities  for  improving  decision  training  programs.  They  suggest  that  three 
aspects  of  the  training  problem  demand  immediate  attention:  training 
specific  skills,  evaluating  the  quality  of  decisions,  and  implementing 
the  knowledge  obtained  through  decision  research.  These  three  areas  are 
essentially  i nterdependent ;  however,  each  entails  sufficiently  different 
research  strategies  to  merit  distinguishing  it  from  the  others. 

With  respect  to  training  specific  skills,  Goodman  et  al.  identify 
several  areas  of  priority.  They  believe  that  judgmental  biases  must  be 
identified  and  the  known  biases  characterized  in  terms  of  underlying 
cognitive  processes.  Research  is  required  to  determine  which  biases  are 
amenable  to  training  and  which  can  only  be  compensated  for  mechanically 
so  that  debiasing  or  bias-avoiding  procedures  can  be  developed  where  ap¬ 
plicable.  These  problems  may  be  most  parsimoniously  attacked  by  looking 
for  common  elements  in  the  decision-making  strategies  used  in  different 
tasks.  These  strategies  are  the  result  of  basic  cognitive  processes,  so 
that  the  interaction  between  basic  cognitive  research  and  cognitive  en- 


2-18 


gineering  will  have  to  be  intensified  when  straightforward  debiasing 
procedures  fail. 

Although  evaluation  is  a  key  element  In  improving  training,  it  is  a  weak 
point  in  current  decision  training  programs.  We  do  not  know  how  to 
evaluate  most  important  decisions.  Without  such  knowledge,  there  is  no 
way  of  assessing  the  value  of  the  various  training  programs  now  offered 
or  the  validity  of  the  claims  made  for  them.  It  may  even  be  that  any 
simple  decision-making  procedure,  however  flimsy  its  axiomatic  basis,  is 
as  good  as  the  most  sophisticated.  The  judgmental  biases  mentioned  ear¬ 
lier  must  also  be  assessed  to  determine  how  much  of  a  difference  they 
make  in  the  optimality  of  decisions.  Some  general  work  on  the  sensi¬ 
tivity  of  decisions  to  bias,  must  be  performed.  Goodman  et  al.  (1976) 
further  suggest  that  ways  be  developed  to  help  people  best  assess  the 
quality  of  their  own  and  their  colleagues  decisions,  and  learn  from 
their  own  experience. 

Current  understanding  and  knowledge  of  decision  analysis  can  be  imple¬ 
mented  to  train  decision  makers  and  improve  the  quality  of  their  perfor¬ 
mance.  For  repeatable  tasks,  the  covert  decision  processes  of  the  ex¬ 
pert  can  be  modeled  and  made  explicit  in  a  way  that  should  be  quite  use¬ 
ful  to  trainees.  In  some  cases,  these  models  will  take  the  form  of 
algebraic  equations.  In  others,  more  complex  models  on  the  order  of 
sequentially  branching  computer  programs,  will  be  necessary.  The  poten¬ 
tial  of  judgment  modeling  for  facilitating  military  and  defense  deci¬ 
sions  is  uni imited. 

Although  Goodman  et  al.  (1976)  place  emphasis  on  the  training  of  specif¬ 
ic  decision  skills,  they  recognize  that  a  common  cognitive  base  may  un¬ 
derlie  various  decision  making  strategies  and  that  a  generalized  ap¬ 
proach  to  decision  training  may  have*  some  merit.  Nickerson  and  Feehrer 
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(1975)  similarly  advocate  an  open-minded  approach  to  the  question  of 
general  as  opposed  to  specific  decision  training.  They  feel  that  an  ef¬ 
fective  decision  maker  in  a  variety  of  situations  needs  some  intellectu¬ 
al  appreciation  for  the  decision-making  process  as  it  is  represented  by 
theoretical  treatments  of  decision  making,  and  some  familiarity  with 
certain  of  the  key  concepts  that  decision  theorists  employ.  Such  key 
concepts  include  a  basic  introduction  to  probability  theory  as  well  as  a 
working  familiarity  with  notion*  of  rationality,  value,  utility, 
mathematical  expectation,  risk,  risk  preferences,  and  so  on.  Failure  to 
provide  an  adequate  grounding  in  theory  might  deprive  the  decision  maker 
of  the  sorts  of  insights  that  lead  to  productive  use  of  available 
decision-aiding  techniques. 

Kanarick  ( 1 969 )  hypothesizes  that  decision  making  can  be  taught  as  a 
skill  which  should  generalize  to  new  situations.  With  respect  to  the 
training  of  Naval  officers,  Kanarick  comes  to  the  same  conclusion  as 
Nickerson  and  Feeher,  namely  that  training  for  specific  job  knowledge 
may  be  supplemented  by  some  general izeo  skill  in  diagnosis  and  action 
selection.  In  this  manner,  the  specifics  of  certain  tactical  situations 
(e.g. ,  capabilities  of  ships  and  weapons,  sensors,  doctrine,  etc.)  would 
be  retained,  while  the  skills  in  decision  making  are  transferred. 
Kanarick  suggests  that  training  decision  making  as  a  skill  early  in  an 
officer's  career  may  provide  him  with  the  basic  tools  necessary  to 
structure  subsequent  decision  problems  so  that  he  can  analyze  them  in 
some  relatively  consistent  and  rational  manner.  To  what  extent  this 
skill  transfers  to  the  operational  situation  and  when  in  an  officer's 
career  this  skill  should  be  trained,  is  a  high-priority  task  for  inves¬ 
tigation. 

With  respect  to  military  systems,  the  development  of  a  generalized  ap¬ 
proach  to  decision  training  would  be  of  particular  value.  One  obvious 
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advantage  would  be  to  facilitate  the  training  of  aircrew  personnel  who 
transition  from  one  aircraft  system  to  another.  Nickerson  and  Feehrer 
(1975)  suggest  that  empirical  research  is  needed  to  determine  whether 
familiarization  with  theoretical  treatments  of  decision  making  will  in 
fact  improve  decision-making  behavior.  They  feel  that  such  training 
will  be  efficacious  for  some  people  performing  certain  types  of  decision 
tasks  but  perhaps  not  for  all  people  or  all  tasks.  One  objective  of 
training  research  should  be  to  identify  those  conditions  under  which 
such  training  would  be  effective  and  those  under  which  it  would  be  a 
waste  of  time.  Clearly,  this  is  an  important  issue,  but  one  which  will 
not  be  resolved  until  evaluative  tools  and  methods  suited  to  decision 
making  and  decision  training  program  evaluation  are  developed  and  ap¬ 
plied. 


2.6  Stress 


The  concept  of  stress  has  been  a  difficult  one  to  define  from  a  theoret¬ 
ical  standpoint  (Appley  and  Trumbull,  1967;  Deese,  1962).  Nevertheless, 
investigators  concerned  with  the  Impact  of  a  variety  of  psychological 
and  environmental  factors  on  performance  both  in  laboratory  and  real- 
world  settings  generally  agree  that  the  introduction  of  variables  com¬ 
monly  recognized  as  extreme  stressors  will  result  in  performance  decre¬ 
ment  or  impairment  (Berkhout,  1970;  Berkun,  1964;  Broadbent,  1971). 

Prophet  (1976,  p.  14)  points  out  that  “There  are  few  performance  en¬ 
vironments  or  situations  that  produce  the  task-time  press,  the  general 
physiological  and  psychological  stress,  and  bodily-harm  threat  as  does 
the  flight  situation..."  The  aircraft  emergency  can  combine  the 
deleterious  physiological  effects  of  a  harsh  physical  environment  with 
the  requirement  for  rapid,  complex  decision  making  under  conditions  of 
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uncertain  information  and  high  personal  risk.  It  would  appear  that  air¬ 
craft  emergencies  are  among  the  ultimate  stressors  for  flying  personnel. 

The  study  of  decision  making  under  the  stress  of  real  life  emergency  si¬ 
tuations  has  not  been  easy.  Berkun  (1964 )  points  out  that  stress,  as  it 
applies  to  the  combat  or  disaster  setting,  involves  an  element  of  threat 
or  personal  risk: 


While  stimulus  overload,  heat,  noise  and  vibration, 
difficult  game  decisions,  and  fine  sensory  dis¬ 
criminations  obviously  produce  a  condition  which 
is  frequently  and  reliably  labeled  "stressed," 
there  is  a  basic  motivation,  drive,  or  attitude  of 
fear  not  ordinarily  manipulated  in  human  factors 
research,  (p.  22) 


It  may  be  that  the  element  of  threat  or  personal  risk  is  the  critical 
factor  that  underlies  the  human  operator's  decision  errors  that  are  in¬ 
volved  in  some  aircraft  accidents.  Wherry  and  Curran  ( 1 966 )  observe 
that  an  operator's  reaction  to  the  threat  of  an  impending  disaster  may 
well  account  for  more  variance  in  performance  among  aviators  and  as¬ 
tronauts  than  their  susceptibility  to  all  the  physical  and  physiological 
stressors  combined.  In  a  similar  vein,  Zavalova  and  Ponomarenko  (1970), 
in  discussing  the  responses  of  pilots  to  emergency  situations,  find  that 


Human  behavior  in  response  to  extreme  factors  may 
be  characterized  by:  (i)  sharp  increase  in  excita¬ 
bility  expressed  in  impulsive  acts,  impairment  and 
loss  of  skills  or  (ii)  inhibition  and  even  the  cessa- 
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tion  of  activity.  Both  types  of  reactions  result  In 
a  disorganization  of  rational  activity  on  the  part  of 
the  individual,  (p.  11) 


Berkun  (1964)  Identifies  criteria  for  studies  which  are  intended  to  as¬ 
sess  the  effects  of  stress  on  performance.  To  predict  from  experimenta¬ 
tion  the  ability  of  men  to  cope  with  real  stresses  requires,  first,  a 
validation  of  the  experimental  situation  as  a  substitute  criterion  for 
uncontrollable  reality.  It  is  argued  that  the  subject  must  cognitively 
perceive  the  situation  as  stressful,  so  that  he  may  react  realistically 
and  not  "as  If."  Simulation  of  a  stressful  environment,  then,  must 
avoid  cues  which  invite  the  subject  to  deliberately  assume  a  role  or 
which  provide  him  with  more  psychological  support  or  sustenance  than  he 
will  receive  in  the  reality  to  which  the  findings  must  generalize. 
Furthermore,  the  task  he  is  to  perform  must  be  meaningful  in  the 
stress-producing  context.  Stressors  which  fulfill  these  requirements 
ought  to  produce  (1)  a  measurable  disturbance  of  performance,  (2)  a  re¬ 
port  of  awareness  of  a  feeling  of  discomfort,  fear,  threat,  or  un¬ 
pleasantness,  and  (i)  a  measurable  perturbation  of  physiological 
(homeostatic)  processes. 

Satisfaction  of  these  criteria  In  a  controlled  research  setting  is  a 
formidable  accomplishment  and,  as  a  result,  our  knowledge  (f  the  effects 
of  stress,  as  defined  by  Berkun,  on  emergency  performance,  In  general, 
and  on  decision  making,  in  particular,  is  mainly  limited  to  anecdotal 
evidence. 

While  stress  is  not  well  understood,  the  need  remains  to  prepare  air¬ 
crews  for  maximum  decision-making  effectiveness  under  emergency  condi¬ 
tions.  A  nunber  of  investigators  recommend  approaches  to  preparing 
scenarios  for  emergency  decision  making.  Bruggink  (1978)  feels  that  the 
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capability  both  for  making  immediate  responses  to  crises  and  for  taking 
slower,  more  reasoned  approaches  to  problems  must  be  developed.  When 
the  unusual  occurs,  there  may  be  only  enough  time  for  an  immediate  reac¬ 
tion,  a  response  that  is  governed  more  by  what  might  be  called  a  sixth 
sense  developed  through  training  and  experience,  than  by  the  process  of 
reasoning.  Additionally,  the  programming  of  simulated  real-world  or¬ 
deals  should  be  encouraged.  A  one-time,  unanticipated  exposure  to  in¬ 
cidents  such  as  an  iced-up  pitot-static  system,  or  total  loss  of  electr¬ 
ical  power,  would  make  a  pilot  more  responsive  to  actual  problems  in 
these  areas.  This  type  of  training  promotes  enlightened  decision  making 
in  unusual  situations  without  down-playing  the  constraints  of  standard 
operating  procedures. 

Zavalova  and  Ponomarenko  (1970)  recommend  mental  rehearsal  as  a  means  to 
improve  readiness  to  respond  to  an  emergency  and  to  cope  with  its  emo¬ 
tional  aspects: 


It  is  known  from  the  psychological  literature  that 
the  mental  representation  of  motor  actions  is 
an  active  mechanism  for  forming  and  perfecting 
occupa.ional  movements.  It  is  important  that  the 
imagery  reconstruction  of  an  emergency  situation 
be  emotionally  colored.  Then  in  an  actual  emergency 
not  only  the  signals  and  modus  operandi  but  also 
the  emotional  sesations  will  be  "familiar," 
thereby  overcome  the  main  psychological  stress 
factor  in  emergencies--surpri$e.  (p,  13) 


A  frequent  theme  in  recommendations  about  emergency  decision  training  is 
that  decision  makers  should  be  given  exposure  to,  or  practice  in,  execu¬ 
tion  of  disadvantageous  decision  alternatives  as  preparation  for  those 
situations  in  which  a  “bad"  choice  is  better  than  total  disaster.  Brug- 
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gink  (1978),  for  example,  recommends  that  pilots  be  taught  to  crash  air¬ 
planes  competently;  that  Is,  to  develop  a  working  knowledge  of  crash 
dynamics  and  techniques  to  minimize  injury,  damage,  and  fire.  Nickerson 
and  Feehrer  (1975)  review  studies  which  Indicate  that: 


...subjects  performed  less  appropriately  when  opera¬ 
ting  at  a  disadvantage  than  when  operating  at  an 
advantage.  One  of  the  conclusions  that  Sldorsky 
and  his  colleagues  drew  from  the  results  of  a  series 
of  studies  was  that  "the  inability  to  analyze  and 
respond  appropriately  In  disadvantageous  situations 
is  a  major  cause  of  poor  performance  in  tactical 
decision  making."  If  this  observation  Is  generally 
valid,  its  Implications  for  tactical  decision  making 
are  clearly  very  significant.  The  Implications  for 
training  are  also  apparent,  namely,  the  need  for 
extensive  decision-making  experience  in  disadvanta¬ 
geous  situations,  (p.  159} 


In  both  of  these  suggestions,  there  is  the  Implication  that  decision 
making  breaks  down  when  all  alternatives  have  negative  outcomes,  but 
that  exposure  and  training  can  overcome  this  impairment. 


Our  limited  understanding  of  the  nature  of  stress  or  Its  effects  makes 
difficult  the  assessment  of  various  proposals  for  training  to  overcome 
the  effects  of  stress  on  emergency  performance.  Brugglnk  Identifies  a 
universal  shortcoming  In  emergency  training  as  "our  inability  to  dupli¬ 
cate  the  unmitigated  stress  of  a  real  or  imagined  threat  to  survival  and 
its  potential  effect  on  Individual  and  team  behavior."  (1978,  p.  5) 


Geese  (1962,  p.  216-217)  discusses  additional  methodological  problems 
associated  with  designing  or  evaluating  training  programs  to  overcome 
the  effects  of  stress.  Although  his  discussion  deals  primarily  with 
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psychomotor  performance.  It  has  relevance  here.  A  major  conclusion  from 
existing  data  Is  that  there  Is  no  simple  relationship  between  stress  and 
performance.  Some  experimental  evidence  led  Oeese  and  Lazarus  (1952)  to 
hypothesize  that  the  effects  of  stress  would  be  deleterious  early  in 
practice  but  not  so  later  in  practice.  This  interpretation  is  equivocal 
however,  because  ar\y  effects  that  are  the  result  of  practice  coulo  be 
attributed  to  adaptation.  In  addition,  performance  on  a  task  is  made  up 
of  several  components,  and  stress  may  affect  each  differentially.  It  is 
impossible  to  say,  generally,  what  effects  will  happen  early  in  training 
compared  with  late  In  training,  or,  Indeed,  whether  or  not  there  will  be 
any  differences  between  early  and  late  practice.  Oeese  (1962)  suggests 
that  an  appropriate  way  to  look  at  the  problem  of  stress  in  training  is 
by  means  of  the  transfer  paradigm,  especially  if  stress  is  Identified  as 
a  stimulus  condition.  Classical  transfer  theory  leads  to  the  prediction 
of  a  decrement  when  stimulus  generalization  occurs,  an  effect  obtained 
by  altering  the  environmental  stimuli.  Thus,  if  stress,  as  a  stimulus, 
is  present  for  either  training  or  performance,  but  not  for  both,  learn¬ 
ing  theory  would  predict  a  decrement,  or  poor  performance,  simply  as  a 
result  of  stimulus  generalization.  Deese,  however,  warns  against  using 
this  concept  In  attempting  to  understand  the  problem  of  stress  and 
training,  because  the  stimulus  character  of  stress  is  ambiguous: 


While  any  stressful  stimulus  has  a  stable  component 
that  gives  rise  to  the  condition  of  stress,  the 
specific  components  of  such  stimuli  vary  widely 
in  their  composition.  Thus,  the  question  of 
stimulus  patterning  must  enter  any  predictions 
about  the  effects  of  stress  made  from  the  point 
of  view  of  the  transfer  paradigm.  Therefore,  It 
appeals  that  the  occasionally  stated  view  that 
stress  should  be  deliberately  Introduced  during 
training  if  it  Is  to  be  expected  during  performance 
is  not  so  sound  as  it  first  appears,  (pp.  216-217) 
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Prophet  (1976)  summarizes  the  current  situation  with  respect  to  training 
for  performance  under  stress  as  follows: 


It  is  clear  that  the  USAF  pilot  must  be  able  to 
perform  reliably  and  effectively  under  conditions 
of  severe  stress.  The  manner  in  which  this 
capability  develops  and  the  extent  to  which  it 
changes  as  a  function  of  conditions  such  as  non¬ 
flying  or  proficiency  flying  episodes,  age,  and 
career,  experience,  and  personal  factors  are  not 
known.  Because  of  the  criticality  of  the  stress 
factor  to  mission  performance,  force  management 
policies  must  be  based  on  sound  knowledge  in  this 
area.  Adequate  mission  performance  requires  more 
than  just  the  requisite  mechanical  skills.  Resis¬ 
tance  to  the  disorganizing  effects  of  stress  must 
be  sufficient  to  permit  the  mechanical  skills  to 
operate  in  effective,  integrated  fashion.  Research 
is  required  to  this  end.  (p.  78) 


2. 7  Summary 


The  foregoing  literature  review  demonstrates  that  there  is  a  growing  in¬ 
terest  in  the  classification  and  analysis  of  decision  making  skills  in  a 
manner  that  is  amenable  to  training  program  development.  It  is  further 
apparent  that  decision  making  is  an  important  and  complex  component  of 
flying  skill,  but  that  the  acquisition  and  maintenance  of  this  and  re¬ 
lated  higher  order  cognitive  skills  are  poorly  understood.  The  limita¬ 
tions  humans  appear  to  exhibit  as  decision  makers  were  reviewed,  togeth¬ 
er  with  certain  suggestions  for  training  programs  designed  to  Improve 
decision  performance.  A  final  area  of  interest,  the  effect  of  stress  on 
emergency  decision  making,  was  recognized  as  important,  but  character¬ 
ized  by  severe  methodological  impediments  to  those  wishing  to  carry  out 
studies  involving  stress. 
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3.  AIRCREW  EMERGENCY  DECISION  TRAINING  CONFERENCE 


3.1  General 


One  of  the  key  efforts  carried  out  in  Year  1  was  the  convening  of  a 
working  conference  to  review  the  state  of  the  art  of  aircrew  emergency 
decision  training;  to  consider  how  current  concepts  in  behavioral  deci¬ 
sion  theory,  safety  research,  and  training  technology  relate  to  aircrew 
emergency  training;  and  to  identify  issues  and  recommendations  for  fu¬ 
ture  work.  This  task  was  seen  as  an  important  step,  together  with  re¬ 
viewing  relevant  background  literature,  in  establishing  a  solid  basis 
for  the  overal 1  research  program. 

Approximately  seventy  individuals  participated  in  the  two  and  one-half 
day  conference  which  was  held  in  San  Francisco  in  late  November  of  1978. 
The  participants  included  representatives  of  the  military  support  com¬ 
munity,  military  contractors,  instrictor  pilots,  and  other  individuals 
concerned  with  aircrew  training,  safety  research,  and  behavioral  deci¬ 
sion  theory. 

3.2  Conference  Program 

The  conference  opened  wit.,  a  statement  of  goals  by  Major  Jack  Thorpe  of 
AFOSR.  Henry  Halff  of  ONR  described  the  relationship  of  OfW  programs  to 
the  emergency  decision  problem.  Next,  position  papers  were  given  on  the 
application  of  decision  theory  to  emergency  situations  by  Ward  Edwards 
(USC)  and  Paul  Slovic  (Perceptronics ,  Inc.).  A  panel  session,  featuring 
military  pilots  and  instructors,  served  to  review  the  emergency  training 
procedures  in  use  by  the  military  services. 
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Current  research  efforts  aimed  at  understanding  and  improving  the  hand¬ 
ling  of  emergency  situations  were  described  by  Major  Duncan  Lieterly  (AF 
HRl ) ;  Joseph  Saleh,  Rosemarie  Hopf-Weichel ,  and  Antonio  Leal  (all  of 
Perceptronics,  Inc.);  Hubert  and  Stuart  Dreyfus  (University  of  Califor¬ 
nia,  Berkeley);  and  Carl  Castore  (Purdue  and  AF  HRL/FT,  Williams  AFB). 

James  Danaher  of  the  National  Transportation  Safety  Board  presented  a 
case  study  of  a  commercial  aircraft  accident  that  occurred  at  St.  Thomas 
in  1976,  in  terms  of  the  decision  tasks  and  decision  sequence  faced  by 
the  pilot.  John  Lauber  and  Renwtck  Curry  (NASA-Ames)  outlined  work  in 
progress  to  study  resource  management  by  commercial  aircrews  using  the 
full  mission  simulation  technique.  They  also  reported  on  the  work  being 
done  at  NASA-Ames  to  study  the  impact  of  cockpit  automation  on  perfor¬ 
mance. 

Robert  Jacobs  (Hughes  Aircraft)  moderated  a  panel  session  in  which  deci¬ 
sion  training  needs  were  reviewed  'ram  a  variety  of  standpoints:  (1) 
aircraft  accident  reporting  and  research  (Richard  Davis,  USC  Safety 
Center),  (2)  simulator  research  and  training  programs  (Elizabeth  Martin, 
AF  HRL/FT,  Williams  AFB),  (3)  instructional  systems  development  (Andy 
Gibbons,  Courseware,  Inc.),  and  (4)  procedural  doctrine  and  the  precrea¬ 
tion  of  emergency  scenarios  (Stan  Roscoe,  University  of  Illinois). 

Current  issues  and  recommendations  for  future  work  were  identified  in 
small  group  sessions  chaired  by  Martin  Tolcott  (ONR),  Anchard  Zeller  (HQ 
AF  Inspection  and  Safety  Center,  SEL),  and  Gary  Klein  (Klein  Associ¬ 
ates).  Throughout  the  meeting  Tony  Modric  (Honeywell)  served  as  a  reac¬ 
tor  to  the  papers  presented,  and  John  Lyman  (UCLA)  ably  summarized  the 
issues  raised  in  his  concluding  review  of  the  meeting. 
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The  presentations  made  at  the  conference  were  synopsized  in  a  report  en¬ 
titled:  Aircrew  Emergency  Decision  Training--A  Conference  Report  which 
was  published  as  a  seperate  report  under  this  contract.  Some  of  the  key 
points  made  and  recommendations  developed  at  the  meetings  are  summarized 
briefly  in  the  following  paragraphs. 

3.3  Critical  Issues 

The  difficulty  of  developing  a  precise  and  universally  accepted  defini¬ 
tion  of  an  emergency  was  recognized  by  all.  There  was  agreement,  howev¬ 
er,  that  a  number  of  factors  are  involved  In  determining  whether  a  par¬ 
ticular  aircraft  situation  would  become  critical  or  not.  These  factors 
include:  crew  experience  and  capability,  environmental  factors  which 
can  ameliorate  or  complicate  a  given  situation,  the  nature  of  the  indi¬ 
vidual  malfunction(s)  involved,  and  the  degree  of  accixnulation  and  com¬ 
pounding  of  malfunctions  and  performance  errors. 

Decision  making  in  emergency  situations  was  discussed  from  a  variety  of 
standpoints.  Several  related  continuua  were  Identified  which  should  be 
considered  by  those  attempting  to  develop  training  systems  for  aircraft 
emergencies,  including: 

(1)  Problem  recognition  versus  problem  diagnosis  (also 
described  as  template  matching  versus  decision  analysis). 

(2)  Response  execution  versus  response  generation,  selection 
and  execution. 

(3)  Standard  procedures  versus  personal  decision  rules. 
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Two  general  schools  of  thought  emerged  at  the  conference,  as  the  above 
continue  suggest--those  who  emphasized  the  importance  of  preplanned 
emergency  situation  management  and  those  who  emphasized  the  variability 
of  emergency  situations  and  the  need  for  a  flexible,  problem-solving  ap¬ 
proach  to  emergency  responses. 

Management  skill  was  identified  as  a  key  element  in  emergency  respond¬ 
ing,  particularly  in  the  multi-person  situation.  Coordination  of  crew 
activities,  information  sources,  and  individual  decisions  within  the 
available  time  frame  and  the  current  mission  context  fall  within  the 
context  of  resource  management,  as  defined  in  the  full -miss ion  simula¬ 
tion  studies  conducted  at  NASA-Ames  Research  Center.  Development  of  the 
skills  of  the  command  pilot  In  managing  both  human  and  technical 
resources  in  an  emergency  was  Identified  as  one  clear  goal  of  emergency 
training  programs. 

Training  programs  were  discussed  from  a  wide  variety  of  standpoints,  in¬ 
cluding  media  and  methods,  sequencing  of  instruction,  the  role  of  the 
instructor,  validation  and  evaluation  of  training  programs,  problems  in 
field  implementation,  and  the  variety  of  audiences  which  emergency  deci¬ 
sion  training  programs  are  required  to  address.  The  role  of  emergency 
scenarios  in  training  program  development  was  reviewed  and  the  impor¬ 
tance  of  validating  the  procedural  doctrine  to  be  applied  in  response  to 
scenarios  was  stressed.  The  need  for  developing  scenarios  which  use 
realistic  cues,  information  rates  and  time  frames  was  pointed  out.  Both 
prospective  and  retrospective  approaches  to  scenario  generation  were 
viewed  as  necessary  to  ensure  that  a  comprehensive  and  relevant  set  of 
training  problems  be  developed. 
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Conclusions 


Some  of  the  more  frequent  recommendations  and  conclusions  which  were 
brought  forth  at  the  conference  include  the  following: 

(1)  Performance  requirements  in  aircraft  emergency  situations 
range  from  rote  responding  to  complex  analysis. 

(2)  Emergency  decision  training  should  address  this  range  of 
requirements. 

(3)  Training  at  all  levels  of  aicrew  proficiency  should  be  con¬ 
sidered,  not  just  at  initial  levels. 

(4)  Decision  theory  concepts  can  be  taught  to  aircrews;  how¬ 
ever,  decision  theory  must  be  linked  to  practical  applica¬ 
tions  to  gain  acceptance  and  use. 

(5)  Areas  of  importance  include:  option  generation,  establish¬ 
ing  utilities,  personal  decision  rules,  and  prepl anning/- 
rehearsal . 

(6)  Instructional  System  Development  ( I SD )  personnel  should  en¬ 
sure  that  the  systems  knowledge  necessary  for  emergency  de¬ 
cisions  is  not  omitted  from  training  for  specific  aircraft 
systems. 

(7)  Training  should  be  carried  out  in  a  manner  that  resembles 
the  real  life  situation  (e.g. ,  via  scenarios)  in  order  to 
faci 1 itate  transfer. 
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(8)  Ancillary  cues  should  be  defined  and  included  in  training 
scenarios/simulations. 

(9)  Design/ development  data  and  field  performance  data  (in¬ 
cidents,  accidents)  should  be  fed  to  ISO  personnel  to  up¬ 
date  training  regularly. 

(10)  Special  attention  should  be  paid  to  teaching  difficult  com¬ 
ponent  skills  individually  and  to  developing  strategies  to 
deal  with  persistent  performance  problems. 

It  was  generally  agreed  that  the  conference  was  successful  in  meeting 
its  goals.  In  particular,  the  meeting  served  to  identify  and  organize 
issues  and  to  bring  together  individuals  with  related  interests.  A 
follow-up  conference  is  proposed  for  the  end  of  Year  2  or  the  beginning 
of  Year  3  to  bring  together  a  similar  group  of  individuals.  The  focus 
of  the  follow-up  conference  will  be  more  specific,  covering  individual 
research  efforts  in  more  depth  and  addressing  selected  issues  in  aircrew 
emergency  decision  training  which  were  identified  at  the  1978  meeting  as 
having  high  priority. 
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4.  CURRENT  APPROACHES  TO  EMERGENCY  TRAINING 


4.1  Introduction 

With  the  exception  of  the  past  year,  the  rate  of  military  aircraft  ac¬ 
cidents  has  shown  a  relatively  steady  decrease  over  the  last  twenty 
years  (Nuvollni,  1979).  Most  of  the  decrease  has  been  attributed  to 
technological  improvements.  Accident  investigation  reports  show  that 
human  error  as  a  cause,  or  as  a  related  factor  in  accidents  has  remained 
uniformly  high.  For  example,  the  National  Transportation  Safety  Board's 
(NTSB)  Annual  Review  of  Aircraft  Accident  Data  for  1977  shows  that  "pi¬ 
lot"  was  a  factor  in  811  of  the  total  accidents,  by  far  the  largest  per¬ 
centage  of  all  factors  listed.  The  next  highest  factor  listed  was  "ter¬ 
rain,"  which  was  a  contributory  cause  in  only  231  of  all  accidents. 
Analogous  findings  are  reported  by  the  United  States  Air  Force.  In  a 
study  of  the  primary  causes  of  major  aircraft  accidents  over  a  ten-year 
period  (1960-1969),  Zeller  and  Thorpe  (1971)  found  that  pilot  error  as  a 
primary  cause  increased  slightly  from  391  to  451,  a  figure  which  does 
not  include  pilot  error  as  a  contributing  cause,  as  the  NTSB  statistics 
do.  The  trend  with  respect  to  pilot  error  has  not  significantly  changed 
during  the  current  decade,  with  almost  501  of  all  U.S.A.F.  accidents 
having  pilot  error  attributed  as  their  primary  cause  (Zeller,  1978). 

During  this  past  year  (1978),  the  overall  accident  rate  in  the  Armed 
Forces,  which  had  been  steadily  decreasing  over  the  past  20  years,  has 
been  reversed,  showing  a  slight,  but  significant  and  puzzling  increase. 
Nuvollni  (1979),  writing  for  "Intercept"  magazine,  describes  a  study 
performed  by  USAF/IG  to  determine  the  nv4iy's"  underlying  this  adverse 
trend.  The  report,  "Change  Pace,"  irtiich  consisted  of  a  detailed 
analysis  of  mishap  data,  an  evaluation  by  the  major  commands,  a  review 
of  the  analysis,  and  the  publication  and  distribution  of  the  results, 
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clearly  showed  that  the  increase  in  the  number  of  destroyed  aircraft  was 
due  to  operational  rather  than  logistical  factors.  Two  causal  factors 
predominated--pi1ot-induced  control  losses  and  controlled  flight  into 
the  ground.  Some  of  the  secondary  causal  factors  identified  were  press¬ 
ing,  distraction,  discipline  breakdown,  lack  of  event  proficiency,  and 
supervision.  Among  the  recommended  actions  was  the  need  for  better 
training  of  pilots,  both  in  terms  of  quality  (realism)  and  quantity 
(more  flying  time).  In  the  present  report,  issues  underlying  the  prob¬ 
lem  of  developing  more  realistic  training  materials  for  decision  making 
in  emergency  situations  are  analyzed. 

4.2  Boldface  and  SET 

While  a  variety  of  approaches  are  currently  employed  in  the  military  to 
train  aircrews  for  emergency  situations,  two  in  particul ar--Boldface  and 
SET--are  of  interest  because  of  their  differences  in  approach  and  in 
their  underlying  theoretical  bases.  A  brief  review  of  these  approaches 
follows  as  a  prelude  to  a  consideration  of  their  theoretical  underpin¬ 
nings. 

Boldface  refers  to  the  large  bold  print  in  flight  manuals  which  identi¬ 
fies  critical  emergency  procedures  and  which  must  be  committed  to 
memory.  In  the  Boldface  approach,  which  has  been  a  standard  training 
method  for  several  decades,  training  in  energency  procedures  emphasizes 
those  relatively  frequent  emergencies  to  which  the  pilot  must  be  able  to 
respond  immediately  without  referring  to  a  checklist.  Boldface  emergen¬ 
cies  are  those  which  are  so  critical  that  there  is  no  time  to  refer  to 
the  pocket  checklist  before  acting.  Typically,  Boldface  procedures  are 
reviewed  thoroughly  and  frequently.  As  an  example,  written  paper  and 
pencil  tests  of  the  complete  set  of  Boldface  procedures  may  be  given 
once  a  month  to  all  flight  personnel;  in  addition,  the  entire  set  of 
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Boldface  procedures  is  divided  into  four  sections,  and  one  section  is 
given  each  week  as  a  mini-Boldface  test.  Then  there  are  also  emergency 
questions  of  the  day,  posted  on  the  flight  schedule  board,  and  discussed 
by  all  personnel.  Altogether,  all  Boldface  procedures  may  be  reviewed 
at  least  three  times  a  month.  Failure  on  Boldface  tests  results  in  the 
loss  of  flight  privileges  for  a  given  period  of  time,  in  general  until 
the  testee  is  qualified  on  reevaluation.  This  time  period  is  variable 
and  is  at  the  discretion  of  the  commanding  officer.  Although  passing  or 
failing  a  Boldface  test  used  to  depend  on  letter-perfect  recall  of  all 
procedures,  the  rules  have  changed  to  some  extent.  Some  minor  errors 
are  occasionally  allowed,  sue.,  as  switching  steps  when  the  order  is  not 
critical  to  the  procedure.  Also,  some  Boldface  procedures  are  occasion¬ 
ally  changed,  and  then  sometimes  switched  back  to  the  original  version. 
Errors  on  tests  reflecting  a  confusion  as  to  which  version  is  currently 
in  force  may  also  be  treated  benevolently.  All  flight  personnel  parti¬ 
cipate  in  these  tests,  not  just  student  pilots.  However,  student  pilots 
receive  other  types  of  training,  both  in  general  procedures  and  in  less 
critical  emergency  procedures. 

Boldface  has  been  criticized  because  performance  evaluation  is  by  means 
of  paper  and  pencil  tests,  but  in  fact.  Boldface  procedures  are  also 
tested  in  the  simulator,  along  with  other  types  of  less  standard  emer¬ 
gency  responses.  Failure  to  perform  the  Boldface  procedures  appropri¬ 
ately  in  the  simulator  also  results  in  loss  of  flight  priviliges.  Per¬ 
formance  on  Boldface  procedures,  then,  is  evaluated  both  on  tests  of 
verbal  recall  and  by  assessing  actual  behavior  in  simulators. 

A  more  serious  criticism  of  Boldface  has  been  presented  by  Thorpe  et  al. 
(1976)  who  point  out  that  Boldface  focuses  on  a  disproportionately  small 
part  of  the  pilot's  overall  task  in  a  given  emergency.  The  ongoing  re¬ 
quirement  to  maintain  aircraft  control,  the  need  to  analyze  the  full 
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emergency  situation,  and  the  importance  of  planning  ahead  for  a  success¬ 
ful  recovery  are  not  part  of  Boldface.  Furthermore,  formal  Boldface 
training  does  not  address  the  need  for  modification  of  responses  when 
situational  factors  make  the  recommended  (Boldface)  responses  inap¬ 
propriate. 

An  additional  problem  is  that  Boldface  procedures  address  single  mal¬ 
functions  only,  while,  in  fact,  many  aircraft  mishaps  are  the  result  of 
a  complex  series  of  events,  any  single  one  of  which  may  not  b?>ve  been 
sufficient  to  cause  an  accident. 

These  three  deficiencies  all  point  to  the  limited  scope  of  Boldface.  In 
a  practical  sense,  however,  the  Boldface  approach  has  several  advan¬ 
tages.  Procedures  are  clear-cut,  easily  communicated  and  evaluated,  and 
once  learned,  are  considered  to  be  highly  stress  resistant.  Among  the 
disadvantages  of  the  Boldface  approach  is  that  general  decision  making 
skills,  such  as  problem  recognition,  information  seeking,  and  generation 
of  alternative  responses,  are  ignored  in  the  training  program. 

The  Situational  Emergency  Training  (SET)  program  (Thorpe  et  al.,  1976) 
was  developed  to  avoid  the  conceptual  and  practical  deficiencies  of  the 
Boldface  approach  and  It  is  used  as  an  alternative  to  Boldface  for  the 
training  of  F-15  pilots  at  Luke  Air  Force  Base.  The  scope  of  SET  in¬ 
cludes: 

(1)  Critical  and  non-critical  emergencies. 

(2)  Consideration  of  all  situational  information  processing  re¬ 
quirements  placed  on  the  pilot. 
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(3)  Development  of  pilot  analytic  and  decision  making  capabili¬ 
ties  which  match  emergency  situation  requirements. 

SET  training  emphasizes  the  use  of  mission  scenarios  to  systematically 
present  the  important  situational  and  psychological  variables  of  air¬ 
craft  emergencies  to  students.  As  an  example,  at  Luke  Air  Force  Base, 
the  general  training  schedule  for  new  F-15  pilots  is  divided  into  two 
phases.  First,  during  the  first  two  weeks,  there  is  an  initial  heavy 
emphasis  on  lectures  which  tapers  off  to  one  hour  a  day.  Students  also 
have  six  sessions  in  the  Cockpit  Procedures  Trainer  (CPT)  using  SET; 
these  sessions,  each  lasting  approximately  one  hour,  orient  the  student 
to  the  simulator  and  to  the  airplane.  There  are  also  several  simulator 
flights  which  include  a  prcc'etermi neo  series  of  emergencies,  along  with 
those  added  by  the  Instructors  as  they  see  fit.  Second,  after  the  first 
two  weeks,  there  is  one  actual  flight  and  one  simulator  flight  daily. 
Occasional  lectures  are  also  given,  and  once  every  two  weeks,  there  is  a 
SET  exercise  in  the  CPT.  SET  scenarios  are  generated  by  instructor  pi¬ 
lots  based  on  accident  reports  and  problems  that  occur  in  the  F-15  and 
then  are  docunented  by  the  Wing  Safety  Officer.  Scenarios  are  short  and 
represent  a  mini-incident  or  problem  with  three  parts: 

(1)  Brief  situation  description  and  presentation  of  critical 
events. 

(2)  List  of  options  (usually  four)  to  select  from. 

(3)  Explanation  of  why  the  various  options  are  appropriate  or 
not. 


A  key  aspect  of  SET  is  that  the  student  and  his  instructor  interact  in  a 
diagnostic  fashion  around  the  decision  making  and  response  elements  of 
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each  mission  emergency  scenario.  In  SET,  individualized  instruction  is 
facilitated  through  the  student- instructor  dialog.  Transfer  to  the  per¬ 
formance  situation  is  enhanced  by  the  use  of  a  CPT  as  part  of  the  in¬ 
structional  setting.  Simulator  training  does  not  provide  this  interac¬ 
tion,  but  on  the  other  hand,  it  supplies  more  realism  by  simulating  ac¬ 
tual  flight  conditions. 

One  problem  with  SET  is  that  the  generation  of  scenarios  is  time- 
consuning  for  the  instructor  pilots  and  formal  guidelines  for  scenario 
generation  do  not  exist.  Hence,  there  is  a  tendency  to  limit  scenario 
generation  to  those  cases  for  which  problems  have  already  arisen.  This 
is  a  practical,  but  limited  approach.  The  SET  1 ibrary  at  Luke  AFB  is 
fairly  small,  a  fact  that  suggests  the  difficulty  of  developing  a  large 
and  comprehensive  library  of  training  scenarios  given  the  current  lack 
of  formal  guidelines. 

An  additional  problem  with  SET,  in  contrast  to  Boldface,  is  performance 
evaluation.  During  a  CPT  session,  instructor  pilots  give  feedback  but 
do  not  formally  grade  problems.  Only  simulator  sessions  are  formally 
graded,  but  in  the  simulator,  separation  of  the  Instructor  from  the  stu¬ 
dent  prohibits  direct  observation  of  the  student’s  behavior,  and  hence, 
immediate  feedback.  Grading  is  subjective,  and  only  based  on  outcomes, 
not  on  ongoing  decision-making  behavior. 

Overall,  however,  SET  represents  a  more  comprehensive  approach  to  emer¬ 
gency  training  than  Boldface,  because  Boldface  only  emphasizes  the 
responses  that  have  to  be  performed  when  discrete  and  specifiable  mal¬ 
functions  occur,  that  is,  when  the  malfunction  and  its  associated  cues 
can  be  predicted  and  accurately  described.  SET,  on  the  other  hand,  has 
the  following  characteristics: 
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(1)  Training  focuses  on  a  functionally  complete  unit,  which  in¬ 
cludes  the  pilot,  the  system,  ground  control,  communica¬ 
tion,  and  situational  factors  such  as  the  weather,  time  of 
day,  and  flight  phase. 

(2)  An  attempt  is  made  to  simulate  the  environment  adequately, 
within  appropriate  constraints,  so  as  to  optimize  transfer 
of  training.  This  includes  an  emphasis  on  situational  de¬ 
tails. 

(3)  SET  emphasizes  flexibility,  where  problem  structuring, 
judgment,  problem  solving,  and  decision  making  play  a  large 
role,  in  contrast  to  the  strict  adherence  to  predetermined 
procedures  characteristic  of  Boldface.  Development  of 
discrimination  skills  is  emphasized  by  including  both 
relevant  and  irrelevant  situational  items  in  the  scenarios. 

(4)  Both  student  and  instructor  assume  active  roles  and 
training/testing  sessions  are  characterized  by  a  fluid  ex¬ 
change  of  information.  The  instructor  has  the  option  to 
probe  the  student's  understanding  of  an  emergency  situation 
through  the  socratic  dialog  as  well  as  by  modifying  the 
parameters  of  the  emergency  problem.  Rapid  evaluation  and 
feedback  provide  the  student  with  timely  reinforcement  and 
knowledge  of  results. 

Although  as  described  above,  SET  and  Boldface  appear  to  represent  two 
antithetical  approaches  to  emergency  training,  in  actuality  they  comple¬ 
ment  each  other,  in  practice  these  two  approaches  or  training  philoso¬ 
phies  are  both  in  use  in  some  fcrm  in  all  aircrew  training  programs, 
although  in  most  programs  SET  may  be  used  in  a  relatively  informal 
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fashion.  Both  approaches  are  clearly  needed  for  aircrew  emergency 
training.  The  range  of  applicability  of  each  to  emergency  training 
should  be  carefully  delineated.  One  way  to  do  this  is  to  examine  the 
cognitive  aspects  of  various  emergency  decision  situations  and  tasks. 

By  doing  so,  the  performance  requirements  and  related  training  processes 
can  be  isolated  and  the  factors  influencing  these  processes  can  be  iden¬ 
tified. 

4.3  Theoretical  F ramework 

The  theoretical  literature  concerning  the  learning  processes  underlying 
a  given  task  can  provide  the  connecting  link  between  the  desired  perfor¬ 
mance  on  a  task  and  the  training  methodology  best  suited  to  obtain  this 
desired  performance.  Alternately,  if  a  training  methodology  is  already 
in  existence,  an  examination  of  its  theoretical  basis  may  help  elucidate 
the  processes  it  is  designed  to  reinforce,  its  limitations,  and  hence 
its  suitability  for  attaining  a  given  training  objective. 

In  the  case  of  Boldface,  wh ich  emphasizes  the  memorization  of  stimuli 
and  responses,  the  theoretical  orientation  best  suited  to  account  for 
the  training  processes  is  clearly  a  combination  of  operant  and  classical 
conditioning.  Boldface  procedures  are  acquired  via  the  process  of  asso¬ 
ciative  learning.  In  the  associative  learning  paradigm,  both  the  stimu¬ 
li  and  the  responses  are  pre-determined,  and  the  only  process  of  in¬ 
terest  is  the  association  that  is  being  formed  between  the  stimuli  and 
the  responses.  Learning  is  viewed  as  a  relatively  passive  process,  gra¬ 
dually  strengthened  by  association  of  stimuli  and  responses  through  re¬ 
peated  trials.  The  stronger  the  association,  the  more  reliable  the 
response--a  desirable  result  in  many  emergency  situations.  There  Is  a 
vast  literature  on  the  factors  vrfiich  affect  the  formation  of  the  associ¬ 
ation,  its  resistance  to  extinction,  the  effects  of  interference  on  for- 
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getting,  the  effects  of  rewards,  and  the  factors  influencing  stimulus 
generalization  and  discrimination.  Much  is  known  in  these  areas  that 
could  be  profitably  applied  to  ensure  that  Boldface  procedures  are  op¬ 
timally  learned,  but  not  overemphasized,  given  their  rather  narrow  focus 
of  appl icabil ity. 

SET  can  best  be  viewed  in  terms  of  a  cognitive,  problem  solving  approach 
to  learning.  Cognitive  learning  theorists  include  such  concepts  as 
“set,"  "attention,"  and  "motivation"  as  determinants  of  behavior.  More 
importantly,  however,  the  concept  of  a  "schema"  as  the  unit  of  analysis 
allows  for  a  much  more  flexible  and  comprehensive  understanding  of 
memory  phenomena  than  the  stimul us- response  (S-R)  unit,  since  a  “schema" 
incorporates  the  influence  of  previous  experiences  and  of  situational 
factors  as  contributing  elements  to  both  the  learning  and  recall 
processes. 

The  concept  of  reinforcement,  central  to  S-R  theories,  implies  that  the 
environment,  rather  than  the  learner,  determines  the  products  of  learn¬ 
ing.  Recent  changes  in  instructional  technology  have  been  brought  about 
by  the  cognitive  movement  in  psychology,  changes  which  put  more  emphasis 
on  the  active  and  constructive  role  of  the  learner.  According  to  Wit- 
trock  (1978),  the  cognitive  approach  leads  to  the  design  of 


...different  treatments  for  different  students  In  different 
situations  to  actively  induce  mental  elaborations  that 
relate  previous  learning  and  schemata  to  stimuli.  In  this 
conception  the  learners  are  active,  responsible,  and 
accountable  for  their  role  in  generative  learning.  That 
theme  expresses  a  centrally  important  part  of  the  cognitive 
movement  in  instruction  and  of  the  state  of  the  art  of 
instruction. 
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...memory,  imagery,  and  other  cognitive  processes  are  now 
being  resurrected  in  the  study  of  instruction  because 
they  are  important  to  the  explanation  and  understanding 
of  human  and  humane  learning.  People  learn  not  only  by 
acting  and  experiencing  the  consequences  of  their  actions 
but  also  by  observing  others,  by  imitating  models,  by 
watching  television,  by  seeing  a  demonstration,  by  dis¬ 
cussing  issues,  even  by  listening  to  a  lecture;  sometimes 
without  practice,  without  reinforcement,  and  without 
overt  action.  Cognitive  elaborations,  such  as  inferences, 
images,  memories  and  analogies  influence  their  learning 
and  understanding.  Learners  often  construct  meaning 
and  create  their  own  reality,  rather  than  responding 
automatically  to  the  sensory  qualities  of  their 
envi ronments. 


The  concept  of  “schema"  makes  it  possible  to  account  for  results  of 
learning  experiments  that  show  a  discontinuity  between  the  objective  in 
puts  of  the  experiment  and  the  recall  performance.  For  S-R  theorists, 
recall  is  assumed  to  be  a  reproduction  of  the  input,  whereas  for  cogni¬ 
tive  theorists,  it  is  a  reconstruction,  dependent  on  previous  experi¬ 
ences  and  on  situational  influences  at  the  time  of  both  input  and  re¬ 
call.  Host  events  to  be  learned  (inputs  into  memory)  or  general  experi 
ences,  are  assimilated  into  the  schemata  and  are  restructured  to  "fit 
the  logical  and  causal  conventions  characteristic  of  the  individuals' 
social  and  intellectual  milieu"  (Bartlett,  1932).  Reconstruction  re¬ 
tains  the  meaning,  but  not  necessarily  the  exact  format,  of  the  input. 


These  two  approaches  are  not  exclusive;  both  types  of  learning  are  pos¬ 
sible  and  can  be  under  the  control  of  the  learner.  In  fact,  by  varying 
instructions,  Podel 1  (1958)  showed  that  the  same  material  could  be 
learned  to  fit  either  an  S-R  explanation  where  recall  is  reproductive, 
or  a  cognitive  explanation  with  recall  being  a  reconstruction  of  the 
learned  material.  More  recently,  Kaufman,  Baron,  and  Kopp  (1966)  found 
that  instructions  "exert  powerful  controlling  influences  over  rates  of 
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response,  influences  which  far  outweighed  the  influences  of  the  rein¬ 
forcing  contingencies  actually  present  in  the  operant  training  condi¬ 
tion"  (p.  243).  The  reality  of  the  schedules  actually  experienced  was 
less  influential  upon  the  learners  than  was  the  reality  described  to 
them  in  the  instructions. 

These  studies  suggest  that  Boldface  and  SET  can  be  used  side-by-side  in 
a  training  program  and  that  the  main  problem  is  to  decide  in  what  situa¬ 
tion  each  is  more  appropriate.  With  respect  to  training  for  emergency 
procedures,  a  study  by  Zavalova  and  Ponomarenko  (1970)  supports  the  no¬ 
tion  that  both  approaches,  SET  and  Boldface,  are  needed  for  optimum  per¬ 
formance.  These  investigators  performed  a  controlled  study  of  pilot 
behavior  involving  an  induced  malfunction  during  an  actual  mission. 

They  found  that  pilots'  responses  could  be  easily  dichotomized  into  a 
first,  immediate  reaction,  and  a  second,  more  deliberate  response.  The 
first  reaction  brought  the  aircraft  under  control.  This  appeared  to  be 
the  equivalent  of  following  a  Boldface  procedure  and  was  described  by 
the  authors  as  an  "unconscious  act  whose  completion  did  not  guarantee 
the  correctness  of  future  actions,"  but  which  did  reduce  "the  danger  of 
disrupting  the  mode  of  operation  right  at  the  beginning."  The  second 
type  of  response  Involved  problem-solving  and  decision-making  processes 
with  the  goal  of  correcting  the  malfunction.  The  success  in  correcting 
the  malfunction  and  the  time  needed  for  it  varied  widely  across  indivi¬ 
duals.  The  results  of  this  study  suggest  that  training  that  Includes 
the  development  of  problem-solving  and  decision-making  skills  in  realis¬ 
tic  settings--an  Intrinsic  characteristic  of  SET--can  considerably  im¬ 
prove  fl  ight  safety. 

Current  instructional  technology  follows  a  behavioristic  orientation  in 
the  sense  that  observable  events  are  emphasized  in  developing  training 
materials.  The  jobs  to  be  trained  for  are  analyzed,  performance  objec- 
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tives  are  specified,  and  training  materials  are  developed  to  conform  as 
closely  as  possible  to  the  criteria  established  for  acceptable  perfor¬ 
mance.  Often,  only  stimuli  and  responses  that  can  be  objectively 
described  are  considered  in  this  scheme.  High  levels  of  transfer  of 
training  can  result,  provided  the  performance  environment  is  clearly 
delineated,  but  generalization  to  other  environments  mey  be  limited. 

It  seems  that  aircrew  emergency  training  which  follows  a  behavioristic 
orientation  can  benefit  from  the  theoretical  developments  of  cognitive 
psychology,  just  as  cognitive  psychology  expanded  the  narrow  focus  of 
S-R  oriented  theories  in  accounting  for  learning  and  memory  processes. 
One  way  to  improve  such  training  is  to  determine  which  cognitive 
processes,  including  decision  making  and  judging,  are  important  in  vari 
ous  emergencies  and  to  modify  training  programs  so  as  to  provide  for 
development  of  these  skills  in  a  comprehensive  fashion. 
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5.  ACCIDENT  REPORT  ANALYSIS 


5. 1  Objective 

The  purpose  of  this  phase  of  the  current  study  was  to  develop  an  under¬ 
standing  of  the  types  of  aircraft  accidents  which  occur  in  Air  Force 
operations,  to  determine  their  relative  frequencies  of  occurrence,  and 
to  identify  some  of  the  major  causes  and  other  factors  which  contribute 
to  such  accidents.  This  information  was  sought  primarily  through  a  re¬ 
view  of  USAF  aircraft  accident  reports.  Of  particular  interest  was  the 
usefulness  of  these  reports  in  reaching  a  better  understanding  of  train¬ 
ing  needs  for  emergency  procedures. 

5.2  Method 


5.2.1  Sample.  Aircraft  accident  reports  represent  a  potentially  use¬ 
ful  source  of  information  about  aircraft  emergencies.  Not  all  emergency 
situations  result  in  accidents;  conversely,  not  every  accident  can  be 
sufficiently  documented  so  as  to  describe  the  emergency  conditions  and 
events  vrtiich  1 ed  up  to  it.  Nevertheless,  reports  of  accident  investiga¬ 
tions  represent  the  most  complete,  stable  and  accessible  source  of  data 
with  respect  to  emergencies.  They  are  of  interest  in  this  study  not 
only  as  a  source  of  information  about  emergencies  which  resulted  in  an 
accident,  but  also  as  an  indirect  source  of  information  about  emergen¬ 
cies  which  were  successfully  resolved. 

The  United  States  Air  Force  requires  that  all  USAF  accidents  and  inci¬ 
dents  be  investigated  by  an  accident  investigation  board.  There  are  six 
categories  of  reports  which  are  submitted  in  sequence,  from  preliminary, 
through  supplemental  and  progress,  to  final  reports.  The  results  of  the 
investigation  are  forwarded  to  commanders  of  operating  echelons,  major 
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commands,  support  commands,  and  HQ  USAF  with  complete  and  detailed  In¬ 
formation  on  all  pertinent  facts  relating  to  the  occurrence.  The  find¬ 
ings  may  serve  as  the  basis  for  modification  of  weapon  systems  and 
changes  in  design  criteria,  and  may  be  used  in  operations  planning,  per¬ 
sonnel  planning,  and  other  staff  actions.  Personnel  at  various  Air 
Force  levels  review  the  report,  evaluate  the  contents,  and  take  appro¬ 
priate  action  (USAF  Accident/Incident  Reporting,  1971).  The  information 
from  the  accident/incident  reports  is  classified  according  to  various 
elements  and  factors  established  in  a  manual  which  is  regularly  updated 
(Aircraft  Accident  and  Incident  Classification  Elements  and  Factors, 
AF1SCM  127-1,  1972),  and  then  is  indexed  and  entered  into  an  automated 
file  located  at  the  HQ  Air  Force  Inspection  and  Safety  Center,  Norton 
Air  Force  Base,  California. 

Several  visits  were  made  to  Norton  AFB  to  become  familiar  with  the  data 
bank,  report  formats,  and  retrieval  procedures.*  As  a  result  of  these 
visits  and  a  preliminary  review  of  a  sample  of  abstracted  reports  and 
other  materials,  all  USAF  aircraft  accident  reports  (in  abstracted  form) 
for  1977  were  selected  as  a  sample.  Some  385  reports  in  total  were 
available  which  represent  all  major  accidents  and  all  minor  accidents 
with  damages  of  at  least  $50,000  which  occurred  in  1977.  Because  an  ac¬ 
cident  investigation  can  take  several  months  to  be  completed  and  the  re¬ 
ports  used  in  this  project  were  obtained  in  mid-1973,  some  reports  were 
still  subject  to  being  updated.  Nevertheless,  the  reports  reviewed 
represent  a  fairly  accurate  overview  of  the  information  available,  in¬ 
cluding  the  type  of  accident,  the  phase  of  operation  during  which  it 
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occurred,  and  the  conditions  under  which  it  occurred,  as  well  as  the 
most  likely  cause  that  produced  the  accident. 

The  385  abstracted  accident  reports  selected  were  each  reviewed  individ¬ 
ually.  Similarities  and  differences  among  reports  were  noted  and  some 
initial  determinations  were  made  about  categorization  of  the  information 
contained  in  the  reports,  as  described  below.  Summaries  of  the  detailed 
review  and  tabulations  of  reports  in  terms  of  major  variables  of  in¬ 
terest  are  presented  in  Section  5.3. 

5.2.2  Accident  Report  Contents.  Aside  from  the  headings  which  clas¬ 
sify  the  accident,  each  report  has  two  parts,  one  labeled  "Description" 
and  one  "Findings."  "Description"  is  a  somewhat  informal  narrative  of 
the  events  preceding  the  incident,  of  the  incident  itself,  and  of  its 
consequences.  "Findings"  is  a  more  structured  account  in  which  an  at¬ 
tempt  is  made  to  attribute  one  or  more  basic  causes  to  the  incident. 
Information  in  the  two  parts  frequently  overlaps. 

Figure  5-1  shows  a  typical  accident  report.  Accidents  are  divided  into 
"Major"  and  "Minor"  depending  on  the  amount  of  damage  and  the  type  of 
injury  incurred.  The  damage  classification  is  either  minor,  major,  or 
destroyed,  and  refers  generally  to  the  amount  of  damage  that  the  air¬ 
craft  incurred.  Injury  classification  can  be  of  four  types:  none, 
minor,  major,  or  fatal.  If  either  the  damage  or  the  injury  classifica¬ 
tion  is  major  or  greater,  the  accident  is  classified  as  a  major  ac¬ 
cident.  Fcr  both  the  injury  and  the  damage  classification,  the  term 
"missing"  is  used  when  the  pilot  and/or  aircraft  were  not  recovered; 
this  is  essentially  the  same  as  "fatal"  and  "destroyed,"  respectively. 
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MINOR  ACCIDENT 


DAMG  CLAS  -  MINOR  INO  CLAS  -  NONE 

TYPE  -  COLLAPSE  OR  RETRACTION  OF  GEAR 
COND-  ARRESTING  BARRIER 
PHASEOPR  -  LANDING  ROLL 
BASIC  -  SHEARED 

DESCRIPTION  F-4E.  ON  LANDING  AIRCRAFT  TOUCHED  DOWN  APPROXIMATELY  500 
FEET  DOWN  RUNWAY.  THE  NOSE  GEAR  STRUT  COMPRESSED  AND 
SHEARED.  AIRCRAFT  FUSELAGE  EXTENDED  THE  BAK-12  BARRIER 
CABLE.  AIRCRAFT  CAME  TO  REST  3800  FEET  DOWN  THE  RUNWAY, 

21  FEET  LEFT  OF  CE’TERLINE.  THE  AC  SHUT  DOWN  THE  AIRCRAFT 
AND  BOTH  CREW  MEMBERS  PERFORMED  EMERGENCY  GROUND  EGRESS. 

FINDINGS  F-4E.  FINDINGS.  (1)  THE  F-4  FLIGHT  MANUAL  ADDRESSED  HIGH 
SINK  RATES  AS  THE  ONLY  SOURCE  OF  NOSE  GEAR  OVERSTRESS. 

(2)  PILOT  LANDED  THE  AIRCRAFT  WITHIN  DESIGN  PARAMETERS  BUT 
THE  STRESS  PLACED  ON  THE  NOSE  GEAR  EXCEEDED  ITS  LOAD  BEARING 
CAPABILITY.  (3)  THE  NOSE  GEAR  STRUT  OUTER  CYLINDER  WAS  NOT 
OF  SUFFICIENT  STRENGTH  TO  WITHSTAND  NORMAL  LANDING  LIMITS 
AND  SHEARED  BECAUSE  OF  MATERIEL  FAILURE  (CAUSE). 


FIGURE  5-1. 

EXAMPLE  OF  CONTENTS  OF  ACCIDENT  REPORT 


Other  classifications  for  each  report  include  "Type,"  “Condition/ 

“Phase  of  Operation,"  and  “Basic  Cause."  Each  may  be  followed  by  one  or 
more  descriptors,  or  it  may  be  left  blank.  (In  this  study,  only  the 
first  description  entered  was  utilized  in  the  tabulations  and  analyses.) 
One  reason  for  leaving  a  category  blank  is  that  accident  reports  are  en¬ 
tered  into  the  central  file  in  various  stages  of  completion,  being  up¬ 
dated  as  more  information  is  obtained,  and  thus,  are  in  some  cases  in¬ 
complete.  In  general,  the  more  serious  the  accident,  the  more  thorough 
and  the  longer  the  investigation,  and  hence  the  more  updates  there  will 
be. 


Analysis 


5.3.1  Basic  Cause.  The  category  “Basic  Cause"  refers  to  the  cause  of 
the  accident,  as  reconstructed  during  the  investigation,  and  is  of  major 
interest  in  the  current  study,  because  it  points  to  the  antecedents  of 
an  emergency  situation.  It  is  clear  from  reading  the  reports  that  the 
assignment  of  cause  is  to  some  extent  arbitrary,  and  that  it  has  not 
been  possible  to  develop  a  completely  standardized  approach  for  this 
category.  A  great  deal  of  subjective  evaluation  must  necessarily  enter 
into  this  assignment  since  the  cause  of  an  accident  has  to  be  inferred 
from  the  findings. 


For  present  purposes,  only  eight  categories  of  basic  causes  were  used, 
although  in  the  reports  reviewed,  this  number  is  larger.  The  eight 
categories  are  the  following: 

(1)  Human  error.  This  includes  errors  attributed  to  the  pilots 
(operator)  as  well  as  other  members  of  the  aircrew.  (This 
category  is  not  one  of  the  basic  causes  listed  in  the 
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manual,  but  was  developed  for  purposes  of  this  study,  as 
explained  below.) 

(2)  FOD  and  bird  strike.  By  far  the  largest  percentage  of 
mishaps  are  the  result  of  FOO  (foreign  object  damage). 

“Bird  strike"  was  included  with  FOO  because  they  often  oc¬ 
cur  together;  i.e.,  a  bird  strike  causes  FOD,  and  the 
results,  in  terms  of  operational  factors,  are  similar.  All 
FOO  and  bird  strike  mishaps  were  categorized  as  minor  ac¬ 
cidents.  They  are  of  some  interest  to  us  because  the  cues 
noted  by  the  aircrew  can  be  the  same  as  for  more  serious 
malfunctions.  For  example,  a  loud  thump  or  engine  vibra¬ 
tion  could  be  the  result  of  FOO,  but  could  also  be  an  indi- 

2 

cation  of  something  more  serious.  In  both  cases,  the  em¬ 
ergency  procedures  to  be  followed  may  be  the  same.  (It 
should  be  noted  that  many  FODs  are  discovered  during  pre- 
or  post-flight  maintenance  inspections  and  the  aircrew  is 
never  even  aware  of  them  at  the  time  of  occurrence.) 

(3)  Improperly  connected  or  installed.  This  refers  to  mainte¬ 
nance  problems,  where  some  part  was  either  not  properly 
connected  or  installed,  resulting  in  a  malfunction. 

(4)  Malfunction/Failure.  A  large  number  of  related  causes 
which  were  individually  listed  in  the  reports  were  col¬ 
lapsed  to  form  the  broader  category  of  “malfunction/ 
failure"  and  Includes  "sheared,"  “materiel  failure,- 


FOO  can  result  in  serious  accidents,  but  no  major  accident  in  1977  was 
attributed  to  FOO. 
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“stress  corrosion,"  "ruptured/burst,"  “chafed/ frayed,"  "me¬ 
tal  fatigue,"  “broken/separated,"  and  "defective." 

(5)  Improper  manufacturing/poor  quality  control.  In  some 
cases,  defective  parts  are  allowed  to  be  placed  in  service, 
resulting  in  equipment  failure. 

(6)  Inadequate  or  poor  design.  This  may  refer  to  any  number  of 
deficiencies  that  were  not  anticipated,  such  as  poorly  hu¬ 
man  factored  positioning  of  switches,  or  use  of  unsuitable 
materials. 

(7)  "Technical  order"  in  error  or  inadequate.  This  refers  to 
errors  or  inadequacies  in  the  procedures  to  follow  in  cases 
of  emergencies. 

(8)  Other.  In  this  category,  subcategories  such  as  "snow/ice," 
"unsafe  surface,"  etc.,  were  included.  In  other  words, 
they  represent  causes  attributed  primarily  to  environmental 
factors.  In  general,  these  factors  contribute  to  acci¬ 
dents,  rather  than  causing  them,  but  in  a  few  cases,  they 
were  listed  as  the  primary  cause. 

Two  other  basic  causes  should  be  mentioned  here.  The  first  is  "dropped 
object."  This  is  listed  as  a  basic  cause  in  several  reports  of  mishaps, 
but  from  reading  the  "description"  and  "findings"  it  was  clear  that  the 
dropped  object  was  often  the  result  of  improper  installation,  or  of  some 
other  malfunction.  For  this  reason,  those  accidents  which  had  "dropped 
object"  as  a  basic  cause,  were  included  in  either  category  3  or  4,  as 
appropriate. 
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A  second  basic  cause  often  used  was  “compressor  stall;"  as  above,  this 
cause  was  the  result  of  some  other  problem,  rather  than  itself  being  a 
basic  cause,  and  the  original  cause  could  usually  be  discovered  by  read¬ 
ing  the  description.  Hence  accidents  which  showed  it  as  a  basic  cause 
were  included  in  a  more  appropriate  category. 

In  many  reports,  no  basic  cause  was  identified.  However,  a  close  read¬ 
ing  of  the  description  and  findings  would  indicate  that  one  of  the  eight 
basic  causes  could  be  assigned,  most  frequently  human  error.  As  noted 
above,  human  error  is  not  a  "basic  cause"  used  by  USAF  coders  at  the 
Norton  Flight  Safety  Center  but  was  included  in  this  review  in  order  to 
identify  accidents  in  which  human  factors  were  involved.  One  such  re¬ 
port  in  which  “human  error-  was  assigned  as  a  cause  after  review  is 
shown  in  the  example  of  Figure  $-2.  "Sasic"  (i.e.,  basic  cause)  is  left 
blank,  but  under  “Findings,"  finding  2  attributes  the  cause  to  "opera¬ 
tions  factor,  operator-:  "The  pilot  attempted  an  unauthorized  low  alti¬ 
tude  rolling  crossover  maneuver  while  in  a  heavy  gross  weight/AFT  CG 
condition  to  change  wing  position  on  the  lead  aircraft,  and  the  aircraft 
stalled  and  departed  controlled  flight."  It  is  clear  that  the  pilot  made 
an  error,  and  thus  caused  an  accident.  The  information  which  is  not 
contained  in  the  report  is  why  the  error  was  committed.  In  all,  46  re¬ 
ports  of  major  accidents  which  had  no  basic  cause  assigned  could  be 
classified  as  due  to  human  error  and  these  are  included  in  the  data  that 
follow. 

Figure  5-3  represents  a  summary  of  the  frequencies  of  occurrence  of 
basic  causes  for  the  87  major  and  the  298  minor  accidents  in  1977,  tabu¬ 
lated  according  to  the  eight  basic  causes  described  above.  Overall,  hu¬ 
man  error  accounted  for  about  1  in  5  accidents;  however,  when  major  ac¬ 
cidents  are  considered  alone,  slightly  more  than  1  in  2  major  accidents 
are  ascribed  to  human  error.  Certain  causes  appear  to  be  more  likely  to 
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MAJOR  ACCIDENT 


DAMG  CLAS  -  DESTROYED  INJ  CLAS  -  NONE 
TYPE  -  SPIN  OR  STALL 
TYPE  -  ABANDONED  AIRCRAFT 
CONO  - 

PHASEOPR  -  LOW  LEVEL  FLIGHT 
BASIC  - 

DESCRIPTION  RF-4C.  THE  MISHAP  AIRCRAFT  WAS  NUMBER  4  IN  A  FOUR-SHIP 

FLIGHT  RETURNING  TO  HOME  BASE.  THE  MISSION  WAS  BRIEFED  TO 
INCLUDE  LOW  ALTITUDE  VISUAL  RECONNAISSANCE.  WHILE 
PROCEEDING  EN  ROUTE,  THE  FLIGHT  ASSUMED  TACTICAL  FORMATION. 
IN  THE  VICINITY  OF  THE  SECOND  PLANNED  TARGET,  WITH  THE 
MISHAP  AIRCRAFT  ON  THE  ELEMENT  LEAD'S  LEFT  WING,  THE 
ELEMENT  LEAD  ENTERED  A  SLOW  LEFT  TURN.  THE  PILOT  OF  THE 
MISHAP  AIRCRAFT  ATTEMPTED  TO  CONTROL  THIS  OVERTAKE  BY  USING 
A  ROLLING  MANEUVER  TO  CROSS  OVER  TO  THE  ELEMENT  LEAD'S 
RIGHT  WING.  DURING  THE  MANEUVERS,  THE  AIRCRAFT  DEPARTED 
CONTROLLED  FLIGHT.  THE  WSO  INITIATED  A  SUCCESSFUL 
DUAL -SEQUENCED  EJECTION  AT  ABOUT  3,600  FEET  AGL.  THE 
AIRCRAFT  WAS  DESTROYED  UPON  IMPACT  WITH  THE  TERRAIN. 

FINDINGS  RF-4C.  FINDING  l.  THE  MISHAP  AIRCRAFT  CONFIGURATION 
PLACED  THE  AIRCRAFT  LONGITUDINAL  STABILITY  IN  AN  AREA 
DEFINED  BY  THE  FLIGHT  MANUAL  AS  BEING  MINIMALLY  ACCEPTABLE 
AND  REQUIRING  SMOOTH  POSITIVE  CONTROL  INPUTS.  FINDING  2. 
CAUSE.  OPERATIONS  FACTOR,  OPERATOR.  THE  PILOT  ATTEMPTED 
AN  UNAUTHORIZED  LOW  ALTITUDE  ROLLING  CROSSOVER  MANEUVER 
WHILE  IN  A  HEAVY  GROSS  WEIGHT/AFT  CG  CONDITION  TO  CHANGE 
WING  POSITION  ON  THE  LEAD  AIRCRAFT,  AND  THE  AIRCRAFT 
STALLED  AND  DEPARTED  CONTROLLED  FLIGHT.  FINDING  3.  THE 
CREW  MEMBERS  EJECTED,  SUSTAINING  NO  SIGNIFICANT  INJURIES: 
THE  AIRCRAFT  WAS  DESTROYED  UPON  GROUND  IMPACT, 


FIGURE  5-2. 

EXAMPLE  OF  REPORT  OF  ACCIDENT  INVOLVING  HUMAN  ERROR 
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CAUSE 

MAJOR 

MINOR 

TOTAL 

"Human  Error" 

46  (52.87)1 

27  (9.06)2 

73  (18.96) 

FOO/Birdstrike 

148  (49.66) 

148  (38.44) 

Malfunction/Fail 

19  (21.84) 

71  (23.83) 

90  (23.38) 

Improper  Connection/ Ins tal lation 

4  (4.6) 

23  (7.72) 

27  (7.01) 

T.O.  in  Error  or  Inadequate 

7  (8.05) 

11  (3.69) 

18  (4.68) 

Inadequate/ Poor  Design 

7  (8.05) 

6  (2.01) 

13  (3.38) 

Improper  Mfg/Poor  Q.C. 

3  (3.45) 

4  (1.34) 

7  (1.82) 

Other 

1  (1.15) 

8  (2.68) 

9  (2.34) 

Total 

87  (IOC*) 

298  (100X) 

385  (100?) 

Percentage  of  all  major  mishaps. 

2 

Percentage  of  all  minor  mishaps. 
3  Percentage  of  all  mishaps. 


FIGURE  5-3. 

1977  AIR  FORCE  ACCIDENTS  CATEGORIZED  BY  BASIC  CAUSE 
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result  In  major  accidents  than  minor  accidents.  These  include 
inadequate/ poor  design,  in  addition  to  human  error.  Causes  which  are 
apparently  more  likely  to  result  in  minor  accidents  than  major  accidents 
include  FOD/birdstrike,  malfunction/failure,  improper  connection/ 
installation,  and  technical  order  in  error. 

Figure  5-4  breaks  down  the  87  major  accidents  by  damage  class  and  injury 
class.  Slightly  over  60%  (52  of  87)  of  these  accidents  resulted  in 
death  or  major  injury,  and  41%  (36  of  87)  resulted  in  both  death  and 
loss  of  aircraft.  Human  error  accounted  for  46  of  these  87  major  acci¬ 
dents,  as  Figure  5-3  shows.  Further  analysis,  not  shown  in  Figures  5-3 
and  5-4,  revealed  that  human  error  could  be  identified  as  a  cause  in  82% 
(31  of  38)  of  the  fatalities  that  occurred,  in  54%  (41  of  76)  of  the  in¬ 
cidents  in  which  aircraft  were  lost,  and  81%  (29  of  36)  of  the  incidents 
in  which  aircraft  and  lives  were  both  lost. 

Of  the  accidents  in  which  aircraft  were  destroyed,  but  no  injuries  oc¬ 
curred,  only  19%  (4  of  21)  were  attributed  to  human  error,  and  of  those 
in  which  no  fatalities  occurred,  only  38%  (15  of  40)  were  attributed  to 
human  error.  The  higher  survival  rate  in  accidents  attributed  to  causes 
other  than  human  error  may  mean  that  system- induced  emergency  situations 
can  be  diagnosed  and  evaluated  more  easily  than  other  types  of  critical 
situations;  in  the  former,  there  is  time  to  make  the  decision  to  eject, 
whereas  in  the  latter,  hixnan  error  often  coincides  with  the  departure 
from  the  boundaries  of  the  performance  envelopes,  and  the  decision  to 
eject  is  made  too  late. 

5.3.2  Phase  of  Operation.  Figure  5-5  presents  a  summary  of  the 
number  of  major  accidents  that  occurred  in  1977  by  phase  of  operation. 
The  categories  employed  correspond  to  those  used  in  the  classification 
manual  with  the  exception  that  a  few  have  been  collapsed  into  larger 
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DAMAGE  CLASS 

INJURY  CLASS 

NUMBER 

PERCENTAGE 

Destroyed 

Fatal 

33 

37.93 

Destroyed  Missing 

Missing  {presumed  dead) 

3 

3.45 

Destroyed 

Major 

13 

.  14.94 

Destroyed 

Minor 

6 

6.90 

Destroyed 

None  (pilots 

parachuted  safely) 

21 

24.14 

None 

Fatal  (parachute 
accidents) 

2 

2.30 

Major 

Major 

1 

1.15 

Major 

None  or  Minor 

8 

9.20 

87 

100% 

FIGURE  5-4. 

MAJOR  ACCIDENTS  CATEGORIZED  BY  DAMAGE  AND 
INJURY  CLASSIFICATION 
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PHASE  Of  OPERATION 


NUMBER 


Engines  running,  not  taxiing  3 
Takeoff  roll  1 
Initial  climb  6 
Prolonged  climb  3 
Inflight  normal;  inflight  other  21 
Inflight  aerobatics  7 
Inflight  refueling  1 
Air-to-air  ordnance  delivery  6 
Air-to-ground  ordnance  delivery  13 
Low-level  flight  11 
Descent;  Flare-out  3 
Landing  approach;  Landing  other  5 
Unpremeditated  go-around  1 
Landing  roll  5 
No  phase  assigned  2 


87 


FIGURE  5-5. 

NUMBER  OF  MAJOR  ACCIDENTS  ACCORDING  TO  PHASE  OF  OPERATION 


5-13 


groupings  (e.g. ,  inflight  normal  and  inflight  other).  Figure  5-5  is  not 
particularly  revealing.  Some  22%  (19  of  B7 )  occurred  during  ordnance 
delivery,  24*  (21  of  87)  occurred  in  normal  or  "other"  inflight  condi¬ 
tions,  and  13*  (11  of  87)  in  low  level  flight.  Additional  tabulations 
of  major  accidents  according  to  phase  of  operation  by  type  or  by  condi¬ 
tion  were  not  found  to  be  of  interest  and  are  not  shown  here. 

5.3.3  Type  of  Accident.  Over  the  years,  descriptors  for  type  have 
been  selected  that  seemed  best  to  describe  an  accident;  new  types  were 
included  as  needed,  and  perhaps  old  types  simply  not  used  anymore  (Zell¬ 
er,  1978).  The  assignment  of  a  descriptor  for  type  may  apply  at  a 
variety  of  points  in  the  sequence  of  events  which  make  up  an  accident, 
as  exemplified  by  the  following  three  types:  “spin  or  stall,"  "colli¬ 
sion,"  and  "abandon  aircraft."  "Spin  or  stall"  can  be  characterized  as 
a  cause  or  as  the  beginning  of  an  emergency  situation;  "collision"  is  a 
result,  and  “abandon  aircraft"  is  an  action  taken  following  an  emergency 
situation.  Thus,  “type"  is  not  always  a  useful  classification  for 
present  purposes  of  identifying  emergency  situations  and  associated 
training  needs,  especially  with  respect  to  major  accidents. 

For  purposes  of  illustration,  major  and  minor  accidents  are  shown  in 
Figure  5-o,  categorized  according  to  type  of  accident.  Nineteen  dif¬ 
ferent  types  of  accidents  are  provided  in  the  manual  for  abstracting  ac¬ 
cident  reports.  Only  15  were  actually  employed  in  1977  as  shown  in  Fig¬ 
ure  5-6. 

By  far  the  greatest  proportion  of  major  accidents,  34*  (30  of  87)  are  of 
the  type  “collision  with  ground  or  water,"  which  reflects  an  outcome  of 
an  emergency  situation,  rather  than  a  type  that  would  be  descriptive  of 
an  emergency  situation.  The  same  is  true  of  the  second  most  frequent 
type,  "abandon  aircraft"  (21*.  or  18  of  87).  The  third  most  frequent 


TYPE 


1.  FI  re/ Explosion  In  the  air 

2.  Flre/Exploslon  on  ground 

3.  Aircraft  collision  In  air 

4.  Collision  with  ground  or  water 

5.  Abandon  aircraft 

6.  Other  Collision 

7.  Spin  or  Stall 

8.  Hard  Landing 

9.  Wheels-up  Landing 

10.  Airframe  Failure 

11.  Collapse  or  Retraction  of  Gear 

12.  Loss  of  Directional  Control 

13.  Loss  of  Olrectlonal  Control  (ground) 

14.  Equipment  jettisoned  Inadvertently 

15.  Loss  of  aircraft  structure  or  equipment 

16.  Ho  type  assigned 


MAJOR  MINOR 


17 
2 
2 

30 

18 

3 

4 
1 
1 
1 
3 

3 


2 


21 

9 

4 

2 

41 

2 

4 

5 

7 
1 

10 

8 
20 

164 


87  298 


FIGURE  5-6. 

COMPARISON  OF  MAJOR  ANO  MINOR  ACCIDENTS  BY  TYPE 
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type,  “fire/explosion  in  the  air"  (20%  or  17  of  87),  does  represent  an 
emergency  situation.  Of  those,  71%  (12  of  17)  have  “abandon  aircraft" 
as  a  secondary  type. 

A  large  proportion  of  minor  mishaps,  55%  (164  of  298)  have  no  type  as¬ 
signed.  One  reason  is  that  they  are  not  usually  investigated  very 
thoroughly,  and  unless  the  “type"  is  obvious  from  the  beginning,  it  is 
simply  left  blank.  Another  reason  mentioned  earlier  is  that  investiga¬ 
tions  take  time  and  the  reports  are  not  necessarily  complete  when  re¬ 
ceived,  although  this  is  more  often  the  case  for  major  accidents  than 
for  minor  accidents. 

As  can  be  seen  from  Figure  5-6,  the  type  "fire/explosion  in  the  air"  was 
relatively  frequent  for  major  as  well  as  for  minor  accidents.  For  this 
reason,  the  reports  of  those  38  accidents  were  examined  in  some  detail 
with  the  idea  of  discovering  some  major  differences  in  the  sequence  of 
events  leading  to  a  major  accident  compared  to  that  of  a  minor  accident. 
Little  information  was  found  in  the  accident  reports  reviewed  to  suggest 
that  differences  in  events  and  aircrew  responses  occurred.  One  possible 
reason  for  this  may  be  that  most  accidents  of  the  type  "fi re/explosion 
in  the  air"  are  caused  by  a  malfunction  of  some  sort  and  that  aircrew 
actions  in  response  to  such  a  situation  are  carefully  prescribed  by  the 
Boldface  procedures. 

5.4  Discussion 


The  sample  of  USAF  accident  reports  reviewed  clearly  supports  the 
widely-held  belief  that  approximately  half  of  the  serious  aircraft  ac¬ 
cidents  which  occur  are  the  result  of  human  error.  Furthermore,  it  ap¬ 
pears  that  human  error  is  more  likely  to  be  associated  with  a  major 
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accident  than  a  minor  accident  if  it  is  identified  at  all  as  a  basic 
cause  of  an  accident. 

Tabulations  of  accidents  by  phase  of  operation  and  type  of  accident  were 
not  particularly  revealing.  They  do  point  to  the  variety  of  accidents 
that  can  occur  and,  when  these  descriptive  categories  are  combined  with 
basic  cause,  they  can  be  helpful  in  identifying  areas  of  training  need. 

Accident  reports  are  typically  very  thorough,  particularly  for  major  ac¬ 
cidents.  The  fundamental  purpose  of  accident  investigation  is  "to 
determine  the  facts,  conditions,  and  circumstances  pertaining  to  the  ac¬ 
cident  with  a  view  to  establishing  the  probable  cause  thereof,  so  that 
appropriate  steps  may  be  taken  to  prevent  a  recurrence  of  the  accident 
and  the  factors  which  led  to  it"  (ICAO,  1970).  It  has  frequently  been 
observed  that  assignment  of  basic  cause  is  a  difficult  task  in  accident 
investigation.  Perhaps  one  of  the  more  valuable  results  of  reviewing  a 
series  of  accident  reports  is  an  appreciation  for  the  complex  chain  of 
events  which  often  underlies  an  accident.  This  is  particularly  true  for 
accidents  involving  human  error  and  human  factors  in  general.  Often, 
the  basic  causes  assigned  to  an  accident  are,  in  fact,  symptoms,  in  con¬ 
trast  to  root  causes,  a  concept  of  considerable  interest  and  importance. 
According  to  Parker  (1978)  for  example,  virtually  all  of  today's  air¬ 
craft  accidents  occur  as  a  result  of  a  repeated  cause.  Technical  defi¬ 
ciencies,  when  discovered,  can  be  corrected  rather  easily,  but  most  of 
the  repeat  causes  include  human  factors  related  deficiencies.  These  are 
more  difficult  to  correct,  because  these  repeat  causes  are  not  usually 
root  causes.  For  example,  one  of  the  most  common  and  serious  repeat 
cause  factors  is  flying  in  adverse  weather  conditions;  it  is  quite  obvi¬ 
ous  that  this  is  not  a  root  cause.  A  root  cause  has  to  answer  the  ques¬ 
tion  “Why?" 
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Peterson  (1975)  defines  root  causes  as  “those  which  would  effect  per¬ 
manent  results  when  corrected,"  but  the  definition  should  include  a 
statement  concerning  the  availability  of  the  means  for  correcting  the 
root  cause.  How  does  one  correct  “flying  in  adverse  weather  condi¬ 
tions"?  The  problem  of  identifying  the  "root  cause"  is  necessarily  very 
complex.  There  are  always  antecedents  to  each  identified  cause.  For 
example,  if  a  heart  attack  is  suspected  as  a  cause  factor,  the  root 
cause  may  have  been  a  deliberate  disregard  of  medical  regulations.  That 
in  itself  may  have  been  the  result  of  inadequate  indoctrination  concern¬ 
ing  the  seriousness  of  concealing  a  disability.  The  determination  of 
root  causes,  when  related  to  hunan  behavior,  is  obviously  difficult. 

With  respect  to  emergency  procedures,  a  pragmatic  approach  may  be  the 
most  useful,  namely  to  single  out  those  cause  factors  which  have  the 
capacity  of  being  manipulated  during  training.  In  this  regard,  the  ma¬ 
terial  available  in  accident  reports  is  useful  because  it  helps  in  gen¬ 
erating  hypotheses  concerning  which  behavioral  elements  appear  important 
and  need  a  greater  emphasis  during  training.  However,  the  review  of  ac¬ 
cident  reports  is  not  sufficient  in  itself  to  clearly  identify  perfor¬ 
mance  problems  or  training  needs.  Additional  detail  is  required  to 
identify  the  events  and  decisions  which  are  involved  in  or  contribute  to 
accidents. 

One  source  of  this  type  of  information  consists  of  direct  interviews 
with  aircrew  members  who  have  experienced  emergencies.  If  the  inter¬ 
views  are  conducted  relatively  soon  after  the  incident,  a  great  deal  of 
information  can  be  obtained  directly  from  the  aircrew,  not  only  with 
respect  to  what  actually  happened,  but  also  concerning  their  thoughts  as 
they  were  trying  to  deal  with  the  emergency  situation.  One  such  inter¬ 
view  is  summarized  as  a  case  study  in  Appendix  A,  entitled  "30  Minutes 
Over  Florida.  "  The  overall  emergency  faced  by  the  pilot  (and  his  stu- 
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den*  pilot)  in  an  S-3  training  flight  is  broken  down  in  Figure  A-l  into 
a  series  of  events  and  dec’sions  which  occurred  from  the  onset  of  the 
emergency  to  its  successful  resolution.  As  Figure  A-l  suggests,  at 
several  points  in  the  overall  series  of  events,  the  incident  could  have 
culminated  in  an  accident,  but  did  not. 

In  summary,  accident  reports  are  a  useful  source  of  data  regarding  the 
settings  and  factors  which  are  associated  with  aircraft  accidents.  To 
the  extent  that  human  error  can  be  associated  with  such  accidents,  it 
should  be  possible  to  identify  improvements  in  aircrew  training  programs 
vrtiich  would  help  to  avoid  or  ameliorate  the  outcomes  of  such  emergen¬ 
cies.  Accident  reports,  themselves,  can  assist  in  identifying  problem 
areas  which  may  be  resolvable  through  improved  training  programs.  Addi¬ 
tional  information  is  clearly  needed,  however,  to  identify  the  specific 
performance  problems  and  hunan  errors  involved  at  a  level  of  detail 
which  can  be  addressed  through  training. 


6.  PRELIMINARY  CLASSIFICATION  OF  EMERGENCY  SITUATIONS 


6. 1  Objective 

The  objective  of  this  phase  of  activity  was  to  identify  a  set  of 
representative  aircraft  emergency  situations  and  classify  them  in  terms 
of  certain  attributes  and  underlying  decision  components.  The  intent 
was  to  provide  an  overview  of  the  types  of  malfunctions  that  need  to  be 
considered  when  developing  training  materials  and  scenarios  for  emergen¬ 
cy  decision  training  programs. 

0.2  Method 


The  information  reported  in  this  chapter  is  the  result  of  interviews 
carried  out  during  site  visits  to  military  bases  and  installations  in 
Southern  California  and  Arizona.  Altogether,  18  pilots  with  varying 
backgrounds  and  experience  levels  were  used  as  sources  of  expert 
knowledge  and  opinion.  They  included  two  senior  pilots  from  the  Office 
of  Naval  Research  in  Pasadena,  one  F - 1 6  test  pilot  from  Edwards  Air 
Force  Base,  four  F-lb  instructor  pilots  from  Luke  Air  Force  Base,  and 
eleven  F - 14  fighter  pilots  from  Miramar  Naval  Air  Station. 

The  interviews  always  began  with  an  explanation  of  the  reasons  and  goals 
of  the  present  study,  followed  by  a  brief  statement  concerning  the  ra¬ 
tionale  for  decision  training  and  a  definition  of  appropriate  terms.  As 
necessary,  pilots  were  given  examples  of  decision  components,  decision 
parameters,  and  attributes  of  malfunctions  to  orient  them  to  specific 
tasks.  In  general ,  pilots  were  asked  to  consider  only  that  aircraft 
system  with  which  they  were  the  most  familiar  when  responding  to  spe¬ 
cific  questions  or  completing  exercises. 
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Data-gathering  efforts  were  broken  down  into  four  stages,  and  methodo¬ 
logical  details  of  each  are  reported  separately  in  the  following  sec¬ 
tions  of  this  chapter. 

6.3  Matrix  of  Emergency  Situations 

Listings  of  emergency  procedures  were  collected  from  the  training  manu¬ 
als  for  several  current  aircraft  systems.  These  listings  were  shown  to 
two  experienced  pilots  who  worked  together  on  this  task.  They  were  in¬ 
structed  as  follows: 


Please  select  those  situations  that  you  consider 
particularly  important,  that  require  decision¬ 
making,  that  appear  complex,  and  for  which  current 
training  methods  do  not  seem  optimal  to  you.  In 
addition,  if  you  can  think  of  situations  that  are 
not  listed,  please  add  them  to  the  list. 


It  became  apparent  that  the  listings  of  emergency  situations  from  flight 
manuals  were  not  sufficient  in  themselves  to  allow  the  pilots  to  charac¬ 
terize  the  situations  as  requested.  Flight  phase  was  found  to  be  neces¬ 
sary  in  order  to  assess  the  importance  of  an  emergency  condition.  As  a 
result,  the  pilots  selected  41  emergency  conditions  (malfunctions)  and  9 
flight  phases.  Figure  6-1  depicts  the  369-cell  matrix  formed  when  the 
41  emergency  conditions  are  crossed  with  the  flight  phases.  The  emer¬ 
gency  conditions  and  most  flight  phases  are  sel f-explanatory.  The  three 
takeoff  conditions  are:  (1)  initial  takeof f--taxi  to  about  110  knots, 

(2)  intermediate  takeoff--110  knots  to  lift-off,  and  (3)  final  takeoff  — 
after  lift-off.  It  should  be  noted  that  not  all  cells  of  the  matrix  are 
meaningful  since  some  malfunctions  already  specify  flight  phase  (e.g., 
drag  chute  deployed  inflight),  and  others  are  not  emergencies  during 
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during  certain  flight  phases  (e.g.,  brake  malfunctions  or  tire  failure 
inf! lght) . 

0.4  Selection  of  Emergency  Situations 

Three  experienced  pilots  were  shown  the  matrix  of  emergency  situations 
and  were  asked  to  assign  a  rating  to  each  cell  for  safety  criticality 
and  for  time  criticality.  The  rating  scales  used  are  shown  in  Figure 
b-2.  Safety  criticality  was  defined  as  the  criticality  of  the  potential 
outcome,  and  the  ratings  correspond  tc  the  classifications  used  by  the 
b.S.  Air  Force  in  reporting  accidents  and  incidents.  Time  criticality 
refers  to  the  amount  of  decision  time  available  when  an  emergency  condi¬ 
tion  is  recognised.  and  the  pilots  suggested  three  ratings:  short, 
long-self,  and  long-assisted.  “Short"  means  that  time  criticality  is 
high,  and  "long"  that  it  is  low.  The  distinction  between  "long-self" 
and  "long-assisted"  simply  refers  to  the  availability  of  assistance  in 
making  a  decision  with  respect  to  the  particular  emergency. 

Figure  6-3  presents  the  results  of  the  rating  procedure  carried  out  by 
the  three  experts.  The  cells  marked  with  an  “X”  represent  the  6/  combi¬ 
nations  of  malfunction  and  flight  phase  which  were  judged  to  be  the  most 
critical  emergency  situations  in  terms  of  safety  criticality  (i.e.,  were 
assigned  ratings  of  1  or  2 ) .  Combinations  for  wtiici.  there  was  no  con¬ 
sensus,  or  nc  rating  was  assigned,  were  eliminated  at  this  stage  from 
further  consideration.  Time  criticality  ratings  were  not  used  because 
it  seemed  important  to  include  high  and  low  time-critical  situations  in 
the  final  Selection. 

The  o;  maf funct lon/f 1 ight  phase  combinations  represent  critical  situa¬ 
tions  following  which  the  aircraft  could  become  damaged  beyond  economi¬ 
cal  repair  and  lives  might  be  lost.  They  are  not  necessarily  complex 
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"FINAL  OUTCOME"  (SAFETY  CRITICALITY) 


1  «  aircraft  and  crew  loss 

2  »  aircraft  loss 

3  ■  major  damage 

4  *  minor  damage 

5  ■  no  damage 


"PROCESS  TIME"  (TIME  CRITICALITY) 

1  •  short 

2  ■  long-self 

3  ■  long-assisted 


FIGURE  6-2. 

RATINGS  USED  FOR  FINAL  OUTCOME  AND  PROCESS  TIME 
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decision  situations,  since,  in  many  cases,  actions  to  be  taken  upon  per¬ 
ception  of  a  malfunction  are  dictated  by  Boldface  procedures  and  do  not 
require  involved  analysis  or  selection  from  among  alternatives.  In  gen¬ 
eral,  a  malfunction  during  ground  operations  or  initial  takeoff  was  not 
considered  by  the  experts  to  lead  to  loss  of  aircraft  or  aircrew. 

Uther  general  findings  had  to  do  with  the  relationship  between  time  and 
safety  criticality,  in  general,  time  criticality  was  rated  high  for  any 
emergency  that  occurs  during  takeoff  or  landing  and  that  also  was  rated 
as  having  high  safety  criticality.  Situations  were  also  rated  as  high 
in  time  criticality  if  they  involved  low  altitude  and  were  rated  as  high 
in  safety  criticality.  Conversely,  for  high  altitude  situations,  even 
if  the  situation  had  a  high  safety  criticality  rating,  low  ratings  of 
time  criticality  were  given,  apparently  because  the  experts  felt  that 
decision  time  is  generally  greater  at  higher  altitudes. 

o. S  Attributes  of  Emergency  Situations 

Unce  the  set  of  emergency  situations  was  pared  down  to  manageable  pro¬ 
portions,  a  somewhat  more  rigorous  analysis  was  possible.  First, 
descriptions  of  the  6/  selected  situations  were  individually  typed  on 
4xo  cards.  Then,  seven  pilots  were  asked  to  rank  order  the  cards  ac¬ 
cording  to  safety  criticality,  time  criticality,  and  current  decision¬ 
making  effectiveness.  For  safety  criticality,  the  pilots  were  asked  to 
rank  situations  in  terms  of  how  dangerous  each  was;  for  time  criticali¬ 
ty.  the  ranking  factor  was  how  much  time  they  had  to  evaluate  the  situa¬ 
tion  and  take  action;  and  for  current  decision-making  effectiveness, 
they  were  asked  to  consider  how  well  they  were  trained  to  deal  with  each 
situation  and  how  effective  they  felt  their  decisions  would  be  if  they 
were  faced  with  it.  The  set  of  cards  was  ordered  twice  by  each  pilot. 


once  each  on  two  of  the  three  attributes.  Each  pilot  received  a  dif¬ 
ferent  combination  and  order  of  attributes. 

Each  situation  was  assigned  a  score  which  was  the  average  of  the  ranks 
assigned  by  the  pilots.  Since  the  pilots  did  not  always  agree  on  how  to 
rank  the  situations,  an  agreement  score  was  calculated  which  was  the 
mean  difference  between  the  raw  rank  scores.  The  scores  are  shown  in 
Figure  6-4.  All  6?  situations  are  listed  in  Figure  6-4,  but  in  some 
cases  no  scores  are  available.  This  is  because  the  pilots  had  the  op¬ 
tion  to  exclude  situations  that  were  not  relevant  to  their  experiences 
and  they  did  so  in  several  cases. 

High  scores  represent  high  time  and  safety  criticality,  and  low 
decision-making  effectiveness.  Although  statistical  measures  of  associ¬ 
ation  were  not  computed  because  relatively  few  date  points  were  ob¬ 
tained,  there  appears  to  be  some  positive  correlation  among  the  three 
attributes.  In  general,  situations  that  have  a  high  safety  criticality 
were  also  those  for  which  the  rankings  suggest  that  there  is  little  time 
to  make  decisions  and  that  current  decision-making  effectiveness  is  low. 

The  distributions  of  mean  ranks  for  each  attribute  are  shown  in  Figure 
o-6.  The  distributions  for  safety  criticality  and  time  criticality  re¬ 
veal  a  broao  distribution  of  scores  and  suggest  that  the  pilots  were 
able  to  discriminate  well  among  the  situations.  The  distribution  for 
decision-making  effectiveness  is  bunched  and  suggests  that  the  pilots 
did  not  see  much  distinction  among  various  situations  with  respect  to 
this  attribute. 

Figure  6-6  presents  the  distributions  of  agreement  scores  for  the  ranks 
produced  for  each  attribute.  The  lower  the  score,  the  better  the  agree¬ 
ment  on  the  rankings.  Inspection  of  Figure  6-6  reveals  that  there  was 
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SITUATION  ATTRIBUTES 
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safety  ciuncAim1 
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SCORE 

TINE  CRITICALITY1 

MEAN  RANK  AGREEMENT 
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OCCISION-MAKING* 
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MEAN  RANK  AGREEMENT 
SCORE 

26  En*ln*  Fir*  (FHS) 

29.5 

21 

11 

14 
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21 

27  Engine  Fir*  (FHF) 

30. S 
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33.5 

1 

39 

21 
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7 

58 
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40.8 

S3 
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FIGURE  6-4.  (CONTINUED) 

MEAN  RANKS  ANO  AGREEMENT  SCORES  FOR  EMERGENCY  SITUATION  ATTRIBUTES 
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ranks  represent  high  criticality. 

2Hlgh  r*nls  r«rt1Mt  Ion  decislon-anking  |ff«tl*rn« 


FIGURE  6-4.  (CONTINUED) 

MEAN  RANKS  AND  AGREEMENT  SCORES  FOR  EMERGENCY  SITUATION  ATTRIBUTES 
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FIGURE  6-6. 

FREQUENCY  DISTRIBUTION  OF  AGREEMENT  SCORES  FOR  SAFETY  CRITICALITY, 
TIME  CRITICALITY  AND  DECISION-MAKING  EFFECTIVENESS 
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good  agreement  among  pilots  for  the  safety  and  time  criticality  rank¬ 
ings,  but  not  for  the  rankings  for  decision-making  effectiveness.  For 
the  latter  attribute,  the  majority  of  situation  rankings  showed  lower 
agreement  than  for  most  time  criticality  and  most  safety  criticality 
scores.  These  data  suggest  that  pilots  are  unable  to  produce  reliable 
rankings  of  their  effectiveness  and  training  in  this  area.  It  may  be 
that  the  effectiveness  of  their  training  varies  widely  or  that  they  are 
simply  not  used  to  making  such  judgments.  The  relatively  low  agreement 
scores  for  these  ranks,  coupled  with  the  lack  of  discrimination  (bunch¬ 
ing)  found,  indicate  that  the  rankings  for  decision-making  effectiveness 
obtained  in  the  present  study  are  of  little  value  at  this  time  and  they 
will  not  be  considered  further.  However,  the  very  contrast  between 
these  rankings  and  those  for  safety  and  time  criticality  has  two  in¬ 
teresting  implications.  On  the  one  hand,  it  suggests  that  the  method 
used  does  produce  reliable  data  for  those  concepts  which  are  well  under¬ 
stood  by  the  pilots,  namely  the  concepts  of  time  and  safety  criticality. 
On  the  other  hand,  the  finding  that  rankings  of  decision-making  effec¬ 
tiveness  were  found  to  be  unreliable  implies  that  decision  making  and 
its  importance  in  dealing  with  emergencies  is  not  sufficiently  em¬ 
phasized  during  training. 

The  frequency  distributions  were  used  to  select  cut-off  points  so  as  tc 
divide  the  rankings  into  three  parts  reflecting  high,  medium,  and  low 
safety  and  time  criticality  scores.  Ranks  between  0  and  19.9  were  la¬ 
beled  low,  2U  to  39.9  were  medium,  and  high  scores  had  rankings  between 
40  and  b7.  Figure  6-7  shows  the  emergency  situations  categorized  in  a 
3x3  matrix  as  either  high,  medium,  or  low  on  each  dimension.  The 
numbering  for  conditions  is  the  same  as  that  shown  in  Figure  b-4.  Only 
ranks  for  which  the  agreement  scores  were  30  or  below  were  considered 
for  this  figure. 
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HIGH 

(0  -  19.9) 

*011* 

(20  -  39.9) 

3.  4.  5.  6.  8.  9. 

10.  14,  28.  52. 

58.  59,  62.  63. 

64,  67 

7.  15.  16.  24, 

25 

2.  II .  22,  23 

21.  26  .  27  .  52. 

53.  54 

12.  36  ,  37  .  38, 

39 

13.  19.  35.  60. 

61 

25 

16 

20.  JO,  31,  32,  15 

33 


FIGURE  6-7. 

CUSTERS  OF  EMERGENCY  SITUATIONS  CATEGORIZED  BY  RANK  ORDER  ON 
SAFETY  AND  TIME  CRITICALITY  (SITUATION  NUMBERS  CORRESPOND  TO 
THOSE  IN  FIGURE  6-4) 
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The  situations  are  fairly  well  distributed  over  all  cells  in  Figure  6-7. 
The  relatively  large  number  of  situations  in  the  high  '.^fety/high  time 
criticality  cell  is  due  to  the  initial  selection  of  emergency  situations 
which  stressed  high  safety  criticality.  No  situation  having  nigh  safety 
criticality  also  has  low  time  criticality;  in  other  words,  for  all  high¬ 
ly  safety-critical  situations,  time  criticality  is  either  high  or  medi¬ 
um.  On  the  other  hand,  there  are  a  few  situations  having  hi"h  time  cri¬ 
ticality,  but  for  which  the  safety  criticality  is  low.  All  five  situa¬ 
tions  in  this  cell  occur  at  high  altitude,  at  either  fast  or  slow  speed. 
Situation  #12  is  stal 1/stagnation,  inflight  high  and  slow.  Situations 
#3b  and  #37  are  low  fuel  pressure,  inflight  high,  fast  and  slow,  and  si¬ 
tuations  #38  and  #3S  are  low  oxygen  with  the  same  flight  phases  as  #36 
and  #37. 

The  distribution  of  the  malfunctions  in  the  matrix  suggests  that  the 
criticality  of  some  malfunctions  is  much  more  dependent  on  flight  phase 
than  others.  For  example,  total  engine  failure  is  almost  always  high  on 
both  safety  and  time  criticality,  although  the  rankings  suggest  that 
there  is  somewhat  more  time  to  react  when  flying  high  and  fast  than  dur¬ 
ing  other  flight  phases.  On  the  other  hand,  the  rankings  for  engine 
stall  or  stagnation  suggest  teat  the  criticality  of  this  malfunction 
depends  much  more  or<  flight  phase  than  that  of  total  engine  failure. 

Both  time  and  safety  criticality  are  high  for  an  engine  stall  or  stagna¬ 
tion  during  final  takeoff,  inflight  low  and  slow,  and  approach  and  land¬ 
ing,  time  criticality  is  still  high,  but  safety  criticality  is  low  for 
the  same  emergency  inflight  high  and  slow. 

The  breakdown  of  emergency  situations  into  related  clusters  as  shown  in 
Figure  b-7  may  be  of  some  value  in  designing  training  strategies  for 
emergency  situations.  The  nature  of  situational  variables  to  be 
represented,  the  permissible  response  time,  and  the  decision  rule  to  be 
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employed,  all  need  to  be  specified  *rtien  a  strategy  for  developing  the 
skill  to  deal  with  a  particular  emergency  is  chosen.  The  degree  of  risk 
and  the  time  available  for  decision  making  would  undoubtedly  influence 
the  selection  of  these  factors  and  the  evaluation  of  the  skills  being 
developed.  To  the  extent  that  groupings  of  situations,  as  are  made  in 
Figure  6-7,  validly  reflect  common  characteristics  of  a  set  of  emergen¬ 
cies,  it  should  be  possible  to  adopt  a  common  training  strategy  which 
takes  these  rommonal ities  into  account. 

6.6  Decision  Types 

The  67  emergency  situations  ranked  in  the  preceding  sections  were  also 
ranked  in  terms  of  two  major  decision  act i vi ties- -problem  structuring 
and  alternative  selection.  The  same  general  technique  as  used  in 
preceding  sections  of  this  chapter  was  used  here.  Six  pilots  served  as 
experts.  They  were  asked  to  rank-order  the  67  emergency  situations  with 
respect  to  the  relative  difficulty  of  either  problem  structuring  or  al¬ 
ternative  selection.  Because  the  task  is  fairly  difficult  and  time  con¬ 
suming,  each  pilot  rank-ordered  the  situations  according  to  only  one  of 
the  two  decision  activities.  The  instructions  to  the  pilots  wero  as 
follows : 


Several  components  are  included  in  the  process  of 
making  a  decision.  We  are  interested  in  how  difficult 
it  Is  to  make  decisions  vrfien  dealing  with  emergency 
situations.  1  will  give  you  a  set  of  cards,  each 
having  a  particular  malfunction  paired  with  a  flight 
phase.  Please  rank  them  In  terms  of  how  difficult 
they  are  with  respect  to  problem  structuring 
(alternative  selection). 
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The  three  pilots  who  ranked  the  cards  according  to  the  difficulty  of 
problem  structuring  were  told  that  problem  structuring  involves  both  a 
recognition  of  the  problem  and  acquisition  of  more  information  concern¬ 
ing  it.  They  were  shown  Figure  6-8  which  contains  a  list  of  questions 
pertaining  to  those  two  components  and  which  was  to  be  used  as  a  guide 
to  assessing  difficulty.  The  three  pilots  who  were  asked  to  rank  situa¬ 
tions  according  to  difficulty  of  alternative  selection  were  told  that 
alternative  selection  involves  defining  the  available  ootlons  and 
selecting  the  best  course  of  action.  They  were  shown  Figure  6-9  which 
contains  relevant  questions  to  be  used  as  a  guide.  All  pilots  were 
asked  to  visualize  each  emergency  situation  when  they  considered  the 
questions  relevant  to  the  difficulty  of  the  decision  type  being  ranked. 

Mean  ranks  and  agreement  scores  were  obtained  in  the  same  manner  as  be¬ 
fore.  The  results  for  problem  structuring  and  alternative  selection  are 
shown  in  Figure  6-10.  The  situation  numbers  are  the  sa..<e  as  those  used 
in  Figure  6-4.  In  certain  cases  pilots  did  not  rank  some  of  the  67  con¬ 
ditions.  Low  scores  on  problem  structuring  and  alternative  selection 
represent  easy  problems,  while  high  scores  represent  difficult  problems. 

The  distributions  of  the  mean  ranks  are  shown  in  Figure  6-11  and  of  the 
agreement  scores  in  Figure  6-12.  As  before,  the  mean  ranks  are  fairly 
well  distributed  over  the  range,  and  the  agreement  scores  are  bunched  up 
at  the  low  end  of  the  distribution.  The  high  correspondence  in  rankings 
by  pilots  is  shown  by  the  low  agreement  scores,  which  suggests  that  the 
task  was  meaningful  and  valid.  The  cut-off  criteria  for  easy  tasks  were 
0  to  19.9,  for  tasks  with  medium  difficulty,  20  to  29.9,  and  for  diffi¬ 
cult  tasks,  they  were  30  to  49.9.  These  criteria  were  the  same  for  the 
problem  structuring  and  the  alternative  selection  tasks. 
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ftCCOQUZIHC  THC  HHtt.P1 


i.  How  eaiy/dlfflcult  It  It  to  recognize  exactly  what  the  problem  It? 

6.  What  in  Ut  varlablet,  or  facton,  which  affoct  the  situation  you 
art  la? 

c.  How  aaty/dlfflcult  It  It  to  Identify  thota  f acton 7 

d.  Art  tha  various  possible  outcomes  of  this  tltuatlon  aaty/dlfflcult 
to  idantlfy? 

a.  It  than  a  big  difference  between  tha  pratant  (emergency)  condition 
and  tha  condition  which  would  prevail  If  thara  wars  no  emergency? 
(Tha  larger  tha  difference,  tha  aore  complex  or  difficult.) 

f.  It  it  aaty/dlfflcult  to  know  exactly  which  factors  should  be  changed 
to  at  to  ellalnate  tha  wergency? 


MTAthlHC  WOM  HtfQAWATlOH  A10UT  TH£  PKOH.D1 


a.  how  aaty/dlfflcult  It  It  to  obtain  wore  Information  about  this 
emergency  condition? 

b.  How  wany  sources  of  information  can  you  rely  on? 

c.  How  reliable  are  those  tour-cat? 

d.  Are  thota  sources  available  to  you? 

a.  It  it  aaty/dlfflcult  to  decide  which  arc  tha  aott  promising  sources 
(tources  *1th  tha  aott  useful  information)? 

f.  It  It  aa')  difficult  to  know  at  what  point  you  have  enough 
Inf*--'.:-.  to  tolva  the  proolam? 

g.  v  it  ,-ha  cost  of  these  sources  (e.g.,  you  may  have  a  vary  reliable 

*.  available  with  vary  good  information,  but  It  might  taka  too 
j  to  obtain  this  Information:  tha  tource  it  good,  but  the  cost  It 
high). 


FIGURE  6-8. 

GUIDELINES  FOR  JUDGING  DIFFICULTY  OF  PROBLEM  STRUCTURING 
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«.  It  It  Miy /difficult  to  select  on*  bttt  court*  of  tctlonT 

6.  How  nny  different  options  do  you  Wink  you  hm  In  solving 
th*  proPlaa?  (Th*  sort  options,  th*  nr*  difficult) 

C.  Ar*  *11  thus  options  pricticil?  (If  nst  of  thM  trt  not 
prttlctl .  U>*  1ml  of  difficulty  go*S  down) 

d.  Ar*  som  of  th*s*  options  contingent  upon  specific  outcows 
or  *w»lronMnt»l  mnts?  If  so.  *r*  th*  contingencies 
rnllstlcl  (Th*  nr*  contingencies,  th*  nr*  difficult) 


SaECTtW6  TX  BEST  QPTIQW 


«.  how  Mty/difflcult  is  it  to  mtuet*  th*  vtrlout  options 
you  hm? 

0.  How  Msy/dlfflcult  is  It  to  select  th*  o*st  on*? 

C.  Mmt  do  you  Ms*  your  selection  on?  It  It  MSy/dlff Icult 
to  dacld*? 

d.  It  It  Mty/difflcult  to  *st*ss  th*  risks  in»olv*d  in  taking 
**ch  of  th*  possitl*  tctlons? 


FIGURE  6-9. 

GUIDELINES  FOR  JUDGING  DIFFICULTY  OF  ALTERNATIVE  SELECTION 
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FIGURE  6-10. 

MEAN  RANK  AND  AGREEMENT  SCORES  FOR  DECISION  COMPONENTS  OF  EMERGENCY 
SITUATIONS  (SITUATION  NUMBERS  CORRESPOND  TO  THOSE  IN  FIGURE  6-4) 
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FIGURE  6-10,  (CONTINUED) 

MEAN  RANK  AND  AGREEMENT  SCORES  FOR  DECISION  COMPONENTS  OF  EMERGENCY 
SITUATIONS  (SITUATION  NUMBERS  CORRESPOND  TO  THOSE  IN  FIGURE  6-4) 
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FIGURE  6-11. 

FREQUENCY  DISTRIBUTION  OF  MEAN  RANKS  FOR  PROBLEM 
STRUCTURING  ANO  ALTERNATIVE  SELECTION 
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As  before,  the  emergency  situations  were  classified  in  a  3x3  matrix  as 
either  high,  medium,  or  low  on  each  dimension,  as  shown  in  Figure  6-13. 
The  situation  numbers  are  again  the  same  as  those  used  in  Figure  6-4. 
Only  those  situations  having  an  agreement  score  of  30  or  better  were  en¬ 
tered  into  the  matrix.  Because  the  agreement  on  this  task  was  slightly 
better  than  on  the  task  requiring  rankings  of  safety  and  time  criticali¬ 
ty,  only  one  score  (on  situation  #19:  stuck  throttle  on  final  takeoff) 
was  eliminated  from  this  matrix  due  to  low  agreement. 

Figure  6-13  shows  how  the  emergency  situations  are  clustered  according 
to  mean  difficulty  rankings  for  the  two  decision-making  activities  each 
situation  involves.  As  with  the  time  and  safety  criticality  rankings, 
the  difficulty  of  the  decisions  involved  for  a  given  emergency  condition 
was  sometimes  independent  of  flight  phase,  and  sometimes  very  much 
dependent  on  the  flight  phase.  For  example,  both  decision  activities 
were  considered  difficult  in  the  case  of  an  engine  fire  in  all  flight 
phases  except  during  initial  and  final  takeoff,  which  were  ranked  to  be 
of  median  difficulty  on  either  problem  structuring  or  alternative  selec¬ 
tion.  In  other  words,  flight  phase  does  not  appear  to  be  an  important 
consideration  in  determining  the  difficulty  of  dealing  with  an  engine 
fire.  This  is  not  the  case  for  the  difficulty  of  stall/departure  deci¬ 
sions  (situations  #63  to  #67)  which  are  distributed  across  5  of  the  9 
cells,  depending  on  the  flight  phase. 

The  groupings  of  these  situations  according  to  the  difficulty  of  the  de¬ 
cision  activities  involved,  as  shown  in  Figure  6-13,  suggests  conclu¬ 
sions  comparable  to  those  made  when  these  situations  were  clustered  ac¬ 
cording  to  safety  and  time  criticality  (Figure  6-7),  namely  that  these 
cluc'ers  may  be  of  some  value  in  designing  training  strategies.  It 
should  be  possible  to  prepare  common  approaches  to  training  for  clusters 
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FIGURE  6-13. 

CLUSTERS  OF  EMERGENCY  SITUATIONS  CATEGORIZED  BY  DIFFICULTY  LEVEL 
ON  PROBLEM  STRUCTURING  AND  ALTERNATIVE  SELECTION  (SITUATION 
NUMBERS  CORRESPOND  TO  THOSE  IN  FIGURE  6-4) 


6-26 


of  specific  emergencies  which  involve  similar  types  of  decision  making 
activities  and  similar  levels  of  difficulty. 

Figure  6-14  summarizes  the  data  obtained  from  all  of  the  expert  inter¬ 
views  with  the  exception  of  data  for  decision-making  effectiveness  which 
was  found  to  be  unreliable.  Based  on  the  difference  between  the  desig¬ 
nations  on  problem  structuring  and  alternative  selection,  a  decision 
type  was  assigned  to  each  situation.  The  decision  types  are  1  (mostly 
problem  structuring),  2  (mostly  alternative  selection),  3  (problem 
structuring  and  alternative  selection  -  complete  decision),  and  B  (most¬ 
ly  Boldface  procedures).  A  decision  type  B  was  assigned  when  both  the 
problem  structuring  and  alternative  selection  were  considered  easy  by 
the  pilots.  If  only  one  of  these  components  was  considered  easy,  the 
decision  type  assigned  was  a  1  or  a  2,  depending  on  which  component  was 
easy.  If  both  components  were  rated  as  medium  or  high  in  difficulty, 
the  decision  type  was  3;  that  is,  the  situation  involved  both  components 
fairly  extensively. 

As  can  be  seen  from  Figure  6-14,  many  situations  require  a  complete  de¬ 
cision.  In  general,  situations  which  were  rated  easy  with  respect  to 
the  two  decision-making  activities  were  also  highly  critical  with 
respect  to  safety  and  time,  suggesting  that  Boldface- 1  ike  procedures  are 
or  should  be  available  to  deal  with  such  dangerous  situations.  At  the 
same  time,  however,  there  are  eight  situations  which  are  rated  high  on 
both  safety  and  time  criticality,  and  which  also  have  ranks  of  medium  or 
high  difficulty  on  the  decision-making  activities.  It  appears  that 
training  which  goes  beyond  a  Boldface-like  approach  and  involves  sub¬ 
stantial  decision  skills  would  be  particularly  desirable  for  such  situa¬ 
tions,  if  these  data  are  at  all  reliable. 


SITUATION  SAFETY  TIME  PROBLEM  ALTERNATIVE  DECISION 

NUttER  CRITICALITY  CRITICALITY  STRUCTURING  SELECTION  TYPE 
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FIGURE  6-14. 

SUMMARY  OF  MEAN  RANKS  ON  SITUATIONS  WITH  AGREEMENT  SCORES  OF  0-30 
(SITUATION  NUMBERS  CORRESPOND  TO  THOSE  IN  FIGURE  6-4) 
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FIGURE  6-14  (CONTINUED). 

SUMMARY  OF  MEAN  RANKS  ON  SITUATIONS  WITH  AGREEMENT  SCORES  OF  0-30 
(SITUATION  NUMBERS  CORRESPOND  TO  THOSE  IN  FIGURE  6-4) 
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FIGURE  6-14  (CONTINUED). 

SUMMARY  OF  MEAN  RANKS  ON  SITUATIONS  WITH  AGREEMENT  SCORES  OF  0-30 
(SITUATION  NUMBERS  CORRESPOND  TO  THOSE  IN  FIGURE  6-4) 


Some  preliminary  guidelines  for  training  are  suggested  by  Figure  6-13 
when  the  decision  type  assignments  for  emergency  situations  are  re¬ 
viewed.  The  situations  which  involve  difficult  or  moderate  levels  of 
problem  structuring  and/or  alternative  selection  would  appear  to  be  can¬ 
didates  for  training  programs  which  emphasize  development  of  decision 
skills.  Situations  low  on  difficulty  for  these  activities  would  be  can¬ 
didates  for  Boldface  training  procedures,  assuming  high  time  and  safety 
criticality  were  involved  also. 

Unfortunately,  data  obtained  to  assess  the  current  decision-making  ef¬ 
fectiveness  of  pilots  were  unreliable  and  can  not  be  used  to  indicate 
whether  current  training  is  or  is  not  effective  with  respect  to  the 
development  of  derision  skills.  The  high  variability  cf  these  data,  to¬ 
gether  with  informal  conversations  with  the  pilots  who  provided  them, 
support  the  impression  that  current  training  programs  do  not  explicitly 
address  decision  skills  in  any  fomal  manner,  but  concentrate  instead  on 
developing  situation  specific  procedural  skills. 

It  was  noted  earlier  that  in  several  cases  the  experts  used  in  this 
study  were  unable  to  agree  on  the  ratings  for  particular  attributes  of  a 
given  emergency  condition.  There  may  be  several  reasons  for  the  lack  of 
agreement  obtained  in  some  of  the  judgments.  First,  the  malfunctions 
may  be  objectively  different  for  different  aircraft.  Second,  pilots  may 
rate  situations  differently  because  they  have  had  different  previous  ex¬ 
periences.  For  instance,  if  a  situation  has  never  actually  been  experi¬ 
enced  by  a  pilot,  his  only  basis  for  rating  it  is  second-hand  knowledge 
obtained  perhaps  from  manuals  or  conversations  with  other  pilots.  On 
the  other  hand,  having  actually  experienced  a  particular  malfunction 
adds  a  different  perspective  to  one's  evaluation  of  it.  Third,  the  Sub¬ 
jective  interpretation  of  »,he  attribute  being  used  for  the  rating  may 
vary.  Fourth,  the  experts'  perceptions  of  the  probability  of  an  outcome 
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as  well  as  of  the  severity  of  the  potential  outcome  of  a  situation  may 

vary  over  malfunctions.  That  is,  one  malfunction  may  be  viewed  as  pos¬ 

sibly  critical,  whereas  another  may  be  seen  as  certain  to  be  critical. 
Differences  in  perception  may  also  reflect  each  expert’s  view  of  the 
likelihood  that  appropriate  corrective  action  can  be  initiated. 

In  the  present  data,  all  these  reasons  are  treated  as  possible  sources 
of  error.  To  eliminate  them,  it  would  be  necessary  to  specify  the  con¬ 
ditions  of  an  emergency  in  great  detail  and  to  obtain  ratings  from  a 
large  number  of  pilots.  This  was  not  a  necessary  condition  at  this 
time,  since  the  purpose  of  this  study  was  only  to  derive  a  relatively 
good  representative  sample  of  emergency  situations.  Nevertheless,  the 
lack  of  agreement  found  in  some  ratings  may  serve  to  indicate  an  under¬ 
lying  difference  between  situations  for  which  there  is  a  consensus  and 

those  for  which  there  is  not.  It  may  be  that  the  number  of  situational 

factors  affecting  the  complexity  and  the  outcome  of  an  emergency  is  re¬ 
flected  inversely  by  the  degree  of  agreement  obtained  on  the  attribute 
values.  High  consensus  in  judging  attributes  suggests  that  the  outcome 
of  an  emergency  is  quite  clearly  defined  by  the  emergency  itself, 
whereas  low  consensus  may  mean  that  the  attributes  of  an  emergency  are 
much  more  situational ly  dependent.  Conversations  with  the  experts  about 
the  ratings  assigned  served  to  support  this  interpretation,  namely,  that 
the  variability  of  ratings  for  certain  emergency  situations  reflected 
the  number  of  situational  contingencies  which  had  to  be  taken  into  ac¬ 
count  in  trying  to  come  up  with  a  single  ranking  for  the  emergency. 

If  attribute  judgments  had  been  obtained  with  a  much  larger  sample  of 
experts,  it  is  possible  that  the  variability  of  estimates,  within  rea¬ 
sonable  limits,  would  serve  as  a  useful  index  of  the  need  to  Include 
contextual  variations  in  training  materials  for  individual  malfunctions. 
For  example,  if  high  consensus  is  obtained,  the  emergency  condition  may 
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not  need  to  be  presented  in  many  different  guises;  on  the  other  hand, 
low  consensus  would  suggest  that  the  outcome  of  an  emergency  is  highly 
dependent  on  situational  events,  so  that  the  low-consensus  malfunction 
should  be  included  for  study  in  many  different  situations.  It  would  be 
interesting  to  test  this  hypothesis  in  a  more  extensive  study.  Another 
point  to  bear  in  mind  is  that  the  nature  of  a  malfunction  or  emergency 
condition  will  vary  with  the  aircraft  involved,  but  that  present  data  do 
not  allow  for  this  refinement,  which  should  also  be  addressed  in  more 
detail  in  an  expanded  study. 

It  is  important  to  bear  in  mind  that  while  this  study  has  only  presented 
single  estimates,  rather  than  distributions  of  the  values  of  the  attri¬ 
butes  assessed,  distributional  information  could  be  obtained  with  a  more 
extensive  effort  than  the  present  study  using  the  same  general  method. 
Distributional  or  base-rate  information  is  a  source  of  information  that 
is  frequently  overlooked  in  making  predictions  (Kahneman  and  Tversky, 
1977}  and  one  that  characterizes  outcomes  in  cases  of  the  same  general 
class,  in  contrast  to  singular  information,  which  describes  specific 
features  of  the  problem  distinguishing  it  from  others  in  the  same  gen¬ 
eral  class.  Such  data  could  be  important  as  a  basis  for  guiding  deci¬ 
sion  training  programs,  because  they  address  a  significant  bias  in  deci¬ 
sion  making.  Kahneman  and  Tversky  (1977)  found  that,  in  general,  people 
give  insufficient  weight  to  distributional  data  and  rely  primarily  on 
singular  information  in  predicting  outcomes  of  decision  situations.  De¬ 
viations  from  optimal  decision  making  may  be  attributed  to  this  bias,  a 
source  of  error  that  could  be  corrected  without  too  much  difficulty  if 
distributional  data  were  available  and  its  value  emphasized. 

The  study  reported  here  dealt  only  with  one  of  several  types  of  emer¬ 
gencies,  namely  emergencies  induced  by  malfunctions.  The  major  reason 
for  this  is  that  malfunction- related  emergencies  can  be  fairly  easily 
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defined  and  that  they  epresent  a  large  set  of  events  which  are  Impor¬ 
tant  from  a  safety  and  training  standpoint.  Decision- related  Informa¬ 
tion  concerning  malfunctions  can  be  more  easily  obtained  during  Inter¬ 
views  than  similar  information  for  emergencies  which  are  more  abstract 
(e.g.,  communications  problems,  operator  error).  The  intention  was  not 
to  enumerate  all  possible  malfunctions  that  occur,  but  rather  to  derive 
an  overview  of  the  types  of  malfunctions  and  their  concomitant  problems 


that  need  to  be  considered  in  developing  a  taxonomy  to  be  used  as  a 
basis  for  generating  training  guidelines. 


7.  EMERGENCY  SITUATION  MODELS  AND  TRAINING  IMPLICATIONS 


7.1  Introduction 


The  preceding  two  chapters  described  efforts  carried  out  to  identify  as¬ 
pects  of  aircraft  emergencies  which  can  be  used  by  training  designers 
and  developers  as  a  backdrop  for  the  preparation  of  aircrew  emergency 
decision  training  programs.  The  data  reported  were  derived  empirically 
from  reviews  of  aircraft  accident  reports  and  from  expert  interviews 
with  military  pilots.  The  present  chapter  takes  a  much  more  theoretical 
orientation  in  that  it  proposes  three  related  models  for  representing 
the  aircraft  emergency  situation.  A  taxonomic  scheme  is  developed  for 
classifying  the  attributes  of  an  emergency  in  light  of  factors  in  the 
representational  models.  Finally,  some  initial  guidelines  for  training 
are  proposed  which  draw  on  the  implications  from  the  models  and  the  pro¬ 
posed  taxonomy. 

7.2  Representative  Mode  1 s  of  the  Emergency  Situation 

In  order  to  lay  the  foundation  to  develop  procedures  for  training  and 
scenario  generation,  the  situations  to  be  trained  for  need  to  be  organ¬ 
ized  in  such  a  way  that  their  components  have  some  relevance  to  the 
desired  performance  and  that  the  behavior  can  be  differential ly  related 
to  various  aspects  of  these  components.  The  following  section  describes 
three  models  which  differentially  relate  emergency  situation  to 
behavioral  variables.  These  three  models  are  (1)  an  objective  event 
model,  corresponding  to  external  events  and  representing  an  objective 
description  of  an  emergency  situation;  (2)  a  decision  model  describing 
the  conscious  processes  needed  to  deal  with  an  emergency,  specifically 
the  components  of  the  decision-making  situation;  and  (3)  a  cognitive 
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model,  describing  a  theoretical  view  of  the  learning  and  memory 
processes  that  take  place  during  training  and  in  actual  emergencies. 

7.2.1  Event  Sequence.  Emergency  situations  can  be  the  result  of  any 
number  of  factors,  including  those  that  are  directly  related  to  the  pi¬ 
lot,  such  as  physiological  disturbances  or  psychological  stress,  commun¬ 
ication  break-downs,  and  so  forth.  Ideally,  all  types  of  emergencies 
should  be  included  in  a  training  program  and  should  be  described  by  an 
event  model.  For  the  time  being,  however,  only  malfunction-induced 
emergencies  are  considered.  Similarly,  causes  or  influences  antecedent 
to  a  malfunction  are  excluded  from  the  present  conceptualization. 

The  event  sequence  represents  an  objective  view  of  the  components  which 
must  be  considered  in  developing  training  guidelines  for  emergency  pro¬ 
cedures.  These  are  shown  in  Figure  7-1.  By  definition,  the  event  se¬ 
quence  begins  with  a  malfunction  that  is  manifested  by  a  pattern  of 
cues.  From  the  pilot's  point  of  view,  it  is  the  information  obtained 
frcm  the  cues  that  starts  the  sequence  of  dealing  with  an  emergency. 

This  information  is  defined  by  the  values  of  the  cue  attributes.  When 
cues  are  perceived  and  interpreted,  they  lead  to  certain  actions  that 
are  described  by  the  decision  model  and  the  cognitive  model  to  be  dis¬ 
cussed  below.  These  actions  may  lead  to  the  identification  of  the  mal¬ 
function  or  to  intermediate  outcomes  that  produce  new  cues  until  the 
problem  is  solved. 

The  cues  must  not  only  be  interpreted  for  an  immediate  decision  with 
respect  to  their  cause  and  a  possible  solution  to  the  emergency,  but 
some  anticipatory  decisions  must  be  made  at  this  time,  predicting  the 
probable  changes  in  the  cues  as  a  function  of  time  and  the  consequences 
thereof.  Both  decision  types  are  described  below,  and  comprise  the  de¬ 
cision  model. 
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FIGURE  7-1. 

EVENT  SEQUENCE  FOR  EMERGENCY  SITUATIONS 
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7-2. if  Decision  Processes.  For  present  purposes,  two  basic  types  of 
decisions  will  be  distinguished.  The  first  is  the  ongoing  decision 
which  requires  immediate  or  continuing  attention.  The  second  is  the  an¬ 
ticipatory  decision  which  may  be  executed  at  a  later  point  in  time.  The 
two  types  of  decisions  are  represented  separately  because  they  have  some 
distinguishing  characteristics,  and  because  there  seems  to  be  an  in¬ 
herent  differe.ee  between  decisions  concerning  the  problems  brought 
about  by  a  malfunction  and  decisions  such  as  those  involving  ejections 
and  abortive  takeoffs.  The  first  type--the  ongoing  decision--can  be 
conceptualized  as  a  classic  decision,  which  includes  problem  structuring 
aspects  as  well  as  alternative  selection  and  evaluation  of  outcomes. 

The  second  type  involves  anticipations  concerning  decisions  that  may 
have  to  be  made  at  some  future  time,  but  because  the  time  frame  for  exe¬ 
cuting  this  type  of  decision  is  so  critical,  the  conditions  and  criteria 
for  executing  it  must  be  predetermined.  The  relevant  components  of 
these  two  types  of  decisions  are  shown  in  Figures  7-2  and  7-3,  respec¬ 
tively. 

Theoretically,  the  ongoing  decision  contains  all  the  components  identi¬ 
fied  in  a  complete  decision  (Figures  7-4  and  7-5),  although  in  practice, 
some  .omponents  may  be  irrelevant  in  specific  cases.  Two  additional 
processes,  hypothesis  generation  and  confidence  rating,  are  Shown  in 
Figure  7-2,  because  these  may  affect  the  manipulation  and  processing  of 
the  subsequent  components.  That  is,  it  is  assumed  that  problem  recogni¬ 
tion  and  structuring  leads  to  the  generation  of  hypotheses  concerning 
the  accuracy  of  this  activity,  and  that  the  amount  of  confidence  the  de¬ 
cision  maker  has  in  how  much  of  the  problem  has  in  fact  been  identified, 
is  likely  to  affect  the  type  and  number  of  alternatives  that  will  be  ex¬ 
amined,  which,  in  turn,  could  influence  subsequent  phases  of  the 
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decision  process.  No  research  is  available  concerning  this  assumption 
and  it  is  only  presented  as  a  possible  consideration  for  future  experi¬ 
mentation. 

Figure  7-2  shows  the  stages  of  information  processing  of  the  ongoing  de¬ 
cision.  When  a  pattern  of  cues  is  perceived  as  deviating  from  the  "nor¬ 
mal”  expected  pattern  of  information,  the  deviation  is  a  signal  that 
something  may  be  wrong.  Problem  recognition  occurs,  which  is  followed 
by  problem  structuring.  The  success  of  problem  structuring  is  a  func¬ 
tion  of  the  attribute  values  of  the  cues  (reliability,  salience,  etc.), 
and  these  values  in  turn  influence  the  degree  of  confidence  a  pilot  has 
in  his  hypotheses  concerning  the  malfunction.  Very  little  is  known  con¬ 
cerning  possible  differences  in  how  alternatives  are  formulated  and 
selected,  given  different  degrees  of  confidence  in  the  problem  structur¬ 
ing  phase.  Confidence  is  influenced  by  the  degree  of  consistency  among 
cues,  and  it  determines  the  amount  of  additional  information  that  is 
sought.  The  more  inconsistency  there  is,  the  more  information  is  needed 
to  resolve  it.  One  bias  that  has  been  described  (Elstein,  Shulman,  and 
Sprafka ,  1978)  is  that  decision-makers  tend  to  seek  evidence  confirming 
their  initial  hypothesis  and  neglect  trying  to  disconfirm  the  hypo¬ 
thesis,  even  if  their  confidence  in  the  initial  hypothesis  is  low. 

When  the  alternatives  have  been  formulated,  hypotheses  concerning  the 
outcomes  under  each  alternative  are  generated  and  a  decision  rule  ap¬ 
plied  Tor  selecting  the  best  alternative.  The  outcomes  following  each 
action  are  tested  against  the  hypotheses  and  the  resultant  information 
is  used  to  start  the  decision  loop  anew  until  the  problem  is  solved  or 
the  mission  ended. 

The  anticipatory  decision  (Figure  7-3)  begins  when  the  cues  have  been 
identified  and  defined,  and  the  problem  has  been  structured.  Confidence 
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in  the  accuracy  of  this  phase  is  not  relevant  at  this  point.  What  is 
relevant  is  that,  given  a  malfunction  and/or  cues,  there  are  certain 
probabilities  associated  with  the  way  these  cues  can  change  over  time 
and  with  the  possibility  of  having  to  abort  a  mission,  eject,  or  perform 
a  forced  landing.  At  the  moment  decisions  such  as  these  have  to  be  exe¬ 
cuted,  it  is  too  late  to  follow  the  rules  of  decision  analysis.  There¬ 
fore,  the  anticipatory  decision  is  one  in  which  the  factors  are  antici¬ 
pated  and  threshold  criteria  for  executing  the  decision  are  preselected. 
Anticipatory  decisions  involve  the  generation  of  hypotheses  concerning 
how  the  available  cues  may  change  over  time  and  the  specification  of 
threshold  values  beyond  which  the  changes  require  immediate  action 
(e.g. ,  ejection).  Criteria  for  decision  execution  may  also  take  new 
cues  or  changes  in  situational  everts  into  account. 

The  differences  in  the  two  classes  of  decisions  are  shown  in  Figure  7-6. 
For  ongoing  decisions  dealing  with  malfunctions,  the  actions  are 
discrete  and  determined  by  the  cues  as  perceived  at  the  moment.  The 
outcomes  are  probabilistic  in  the  sense  that  they  may  depend  on  factors 
that  are  not  predictable,  or  they  may  depend  on  the  accuracy  of  the 
problem  recognition  and  structuring  pha-.e;  the  outcomes  are  also  proba¬ 
bilistic  because  the  estimated  utilities  determine  the  selection  of  the 
alternative  actions.  For  anticipatory  decisions,  on  the  other  hand,  the 
actions  are  probabilistic  and  the  outcomes  are  determined.  The  actions 
are  probabilistic  because  they  depend  on  a  critical  threshold  that  may 
or  may  not  be  reached,  and  on  the  anticipated  changes  in  the  cues.  Once 
the  threshold  is  reached,  the  outcomes  are  well-defined  since  they  are  a 
function  of  the  anticipatory  decision.  The  outccme  utilities  do  not  af¬ 
fect  the  event  alternatives  since  these  are  determined  by  extraneous 
factors  that  are  not  under  the  control  of  the  pilot.  The  utilities  are 
known,  and  the  outcome  is  not  a  function  of  the  pilot's  decisions 
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(although  it  is  related  to  the  type  of  malfunction  that  occurred  and  to 
how  the  pilot  dealt  with  it). 

The  critical  problem  for  anticipatory  decisions  is  to  recognize  the 
changes  in  the  relevant  cues  and  the  degree  of  changes  that  can  be 
tolerated*  Expert  data  obtained  during  interviews  suggest  that  there  is 
no  objective  way  to  cefine  these  changes,  that  they  are  a  function  of 
experience  and  "feel"  for  the  aircraft.  These  same  data  also  suggest 
that,  while  the  threshold  criteria  may  be  successfully  predetermined, 
the  problem  lies  in  the  actual  execution  of  the  decision,  especially  in 
the  case  of  a  decision  to  eject.  According  to  one  expert  pilot,  there 
may  be  an  interesting  difference  in  this  respect  between  experienced  and 
inexperienced  pilots.  Although  inexperienced  pilots  may  know  how  to  set 
the  criteria,  they  may  not  follow  through  with  their  decisions  or,  for 
reasons  such  as  lack  of  confidence  and  fear  of  repercussions,  they  may 
change  their  mind  at  the  last  minute.  Experienced  pilots  tend  to  make 
the  opposite  error:  once  they  make  an  anticipatory  decision,  they  tend 
to  stay  with  it,  even  when  new  Information  is  obtained  that  would  sug¬ 
gest  a  change.  From  a  training  point  of  view,  it  would  seem  desirable 
to  investigate  the  possible  reasons  for  this  experience- related  differ¬ 
ence,  and  find  out  if  it  is  possible  to  Influence  both  tendenc1es--the 
one  that  delays  the  execution  of  the  decision  and  the  one  that  ignores 
new  information. 

The  decision  processes  represented  here  can  be  trained  directly,  because 
these  are  processes  of  which  the  pilot  is  aware.  Knowledge  structures 
can  be  developed  that  Include  the  relevant  elements  of  decision  making 
and  the  decision  rules  appropriate  in  specified  circunstances. 

7.2.3  Cognitive  Processes.  The  cognitive  processes  model  is  shown  in 
Figure  7-/.  It  is  loosely  patterned  after  the  classic  TOTE  (test- 
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FIGURE  7-7. 

COGNITIVE  PROCESS  MODEL 


operate-test-exit)  unit  of  Miller,  Galanter,  and  Pribram  (1960)  that  as¬ 
sumes  a  feedback  loop,  whereby  inputs  from  the  environment  are  tested 
for  congruity  against  some  established  criteria.  If  the  test  is  posi¬ 
tive,  the  input  information  is  congruous  with  the  information  available 
in  active  memory,  and  an  action  can  he  taken.  If  the  test  leads  to  in¬ 
congruity,  additional  information  from  long-term  memory  (LTM)  must  be 
activated  until  the  results  of  the  test  are  congruous. 

In  the  present  model,  some  additional  assumptions  are  made  concerning 
the  organization  of  LTM  and  the  processes  whereby  items  are  entered  into 
active  memory.  LTM  is  assumed  to  consist  of  nunerous  systems,  each 
representing  a  meaningful  cluster  of  related  information.  These 
“representational  systems"  are  analogous  to  the  concept  of  "schema" 
(e.g.,  Bartlett,  1932;  Hebb,  1949)  or  to  Lashley's  (1958)  trace  systems. 
All  permanent  events  in  memory  belong  to  one  system  or  another,  but  sys¬ 
tems  may  overlap  in  varying  degrees,  share  subsets  of  events,  or  be  re¬ 
latively  autonomous.  Also,  systems  vary  on  a  number  of  dimensions,  such 
as  size  (how  much  is  known  about  a  subject  matter),  stability  (how  reli¬ 
ably  the  knowledge  can  be  retrieved),  and  complexity  (how  detailed  the 
knowledge  is).  Both  content-specific  and  procedural  knowledge  are  in¬ 
cluded  in  a  representational  system,  so  that  at  this  level  no  differen¬ 
tiation  is  made  between  structuring  the  problem  in  response  to  cues  and 
dealing  actively  with  an  emergency. 

It  is  assumed  that  information  in  LTM  Is  latent  and  that  it  must  first 
be  entered  into  active  memory  before  it  can  lie  processed  (Lashley, 

1958).  This  implies  that  systems  can  only  be  altered  when  they  are  ac¬ 
tive,  thus,  learning  (increasing  the  size  of  a  system)  and  forgetting 
(decreasing  the  size  of  a  system  or  its  reliability)  can  only  take  place 
while  the  appropriate  system  is  in  active  memory.  It  also  implies  that 
incoming  stimuli  (e.g.,  cues)  can  only  be  understood  with  respect  to  ln- 
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formation  belonging  to  systems  that  ere  active  at  the  time.  It  is  pos¬ 
sible  for  several  systems  to  be  active  at  the  same  time,  depending  on 
their  size  and  complexity,  and  a  cue  can  be  represented  in  more  than  one 
system  at  the  same  time;  but  in  general,  the  most  salient  cue  will  acti¬ 
vate  the  system  that  has  its  best  representation.  As  a  simplification, 
it  is  assumed  that  when  relevant  cues  from  the  environment  are  per¬ 
ceived,  they  serve  to  activate  the  representational  system  that  contains 
the  information  necessary  to  process  the  cue  or  to  understand  its  mean¬ 
ing.  The  meaning  of  a  cue  is  always  understood  with  respect  to  the  sys¬ 
tem  that  is  active  at  the  time,  just  as  a  homonym  is  interpreted  with 
respect  to  its  immediate  context. 

There  are  two  ways  that  a  system  can  be  activated.  The  first  was  al¬ 
ready  mentioned,  namely,  through  environmental  stimuli  that  are  per¬ 
ceived  as  being  Incongruous  with  those  systems  that  are  active  at  the 
time.  This  implies  of  course,  that  as  long  as  one  is  conscious,  there 
is  always  some  system  that  is  active;  the  problem  as  far  as  information 
processing  is  concerned,  is  how  to  switch  from  one  system  to  another. 

The  second  way  is  internally.  If  an  element  in  an  active  system  also 
belongs  to  another  (inactive)  system,  that  element  has  the  potential  of 
activating  the  second  system. 

These  minimal  assumptions  are  sufficient  to  justify  some  of  the  training 
implications  to  be  derived.  A  more  comprehensive  model  and  its  implica¬ 
tions  for  learning  processes,  and  hence,  for  structuring  training  ma¬ 
terials,  is  described  in  Hopf-Weichel  (1976)  and  Weichel  (1972).  It 
serves  as  a  basis  for  developing  hypotheses  concerning  how  to  maximize 
the  probability  that  relevant  information  will  be  available  when  needed. 

This  conceptualization  makes  it  possible  to  deal  with  the  notion  of 
"templates, “  their  role  in  training  and  in  dealing  with  emergencies,  and 
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their  limitations.  Templates  may  be  defined  as  preplanned  responses  to 
emergencies.  In  the  present  formulation,  a  template  is  a  special  case 
of  a  representational  system,  which  can  only  be  activated  when  the  pat¬ 
tern  of  the  environmental  stimuli  matches  all  the  elements  in  the  sys¬ 
tem.  It  is  extremely  well -rehearsed  and  rigid  in  the  sense  that  indivi¬ 
dual  external  elements  are  not  likely  to  activate  other  systems.  In 
other  words,  the  correspondence  between  external  events  and  the  elements 
of  the  template  is  highly  specific,  and  the  system  itself  is  relatively 
autonomous,  so  that  it  does  not  tend  to  activate  other  systems,  and  gen¬ 
erally  it  can  be  activated  only  by  a  specified  configuration  of  external 
stimuli.  In  this  way,  responses  to  these  external  stimuli  are  highly 
reliable  and  stress-resistant. 

There  will  be  times  when  templates  become  activated  when  in  fact  they  do 
not  represent  the  best  solution  to  the  situation.  The  problem  for 
training  is  to  teach  pilots  to  recognize  when  the  templates  are  applica¬ 
ble  and  how  to  activate  the  appropriate  representational  systems  for 
dealing  with  unplanned  emergencies  when  templates  are  not  appropriate. 

7.3  Basic  Taxonomic  Structure 

7.3.1  Introduction.  Webster  defines  “taxonomy"  as  "the  study  of  the 
general  principles  of  scientific  classification,"  or  alternately,  as  a 
"classification;  specifically,  the  orderly  classification  of  plants  and 
animals  according  to  their  presumed  natural  relationships."  Implicit  in 
the  above  is  the  idea  that  a  taxonomy  has  some  purpose  and  that 
classifications  are  organized  according  to  some  underlying  principle.  A 
revised  definition,  therefore,  is  proposed:  A  taxonomy  is  a  way  to  or¬ 
ganize  components  of  a  subject  matter  according  to  an  underlying  princi¬ 
ple  which  is  used  for  some  purpose. 
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Four  basic  components  have  been  identified  as  particularly  important  in 
describing  an  emergency  situation  and  in  developing  training  guidelines 
appropriate  to  those  situations.  The  basic  components  include  the  si¬ 
tuation  as  a  whole,  the  malfunctions,  the  cues  arising  from  the  malfunc¬ 
tion,  and  the  behavior  of  the  pilot.  Each  component  is  characterized  by 
a  set  of  attributes  having  two  or  more  values.  These  attributes  identi¬ 
fy  the  qualities  of  the  components  that  are  important  in  differentiating 
among  training  approaches.  The  values  assigned  to  the  attributes  are  to 
a  large  extent  determined  by  practical  considerations  and  can  easily  be 
changed  as  the  need  arises.  For  example,  if  one  of  the  attributes  of 
behavior  is  "programmable"  and  two  values  (yes/no)  are  assigned  to  it, 
it  only  means  that  for  the  time  being  these  two  values  are  considered  a 
sufficient  breakdown  in  terms  of  the  training  guidelines  to  be 
developed.  However,  it  might  be  necessary  to  assign  additional  values 
(e.g.,  partially  programmable)  to  this  attribute  if  the  taxonomy  were 
applied  to  a  specific  training  situation. 

The  structure  of  the  taxonomy  is  not  rigid.  It  is  a  preliminary  attempt 
to  categorize  emergency  situations  in  terms  of  the  training  needs  that 
are  anticipated.  These  needs  will  differ,  depending  on  how  specific  and 
predictable  the  behavior  of  the  pilot  is,  and  whether  the  behavior  in¬ 
cludes  psychomotor  as  well  as  procedural,  cognitive,  and  decision-making 
behavior.  As  such,  the  taxonomy  represents  a  general  scheme  for  guiding 
training  development. 

7.3.2  Components  and  Their  Attributes.  The  components  of  the  taxono¬ 
my  represent  the  major  categories  that  have  to  be  described  in  terms  of 
a  set  of  attribute  values.  In  the  present  context,  attributes  are  not 
additive  components  of  utility,  rather  they  represent  general  features, 
used  as  differentiators  of  situational  demands  and  pointers  to  subse¬ 
quent  training  requirements.  The  values  assigned  to  the  attributes  may 
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be  either  quantitative  or  qualitative,  and  serve  to  characterize  dif¬ 
ferent  types  of  emergency  situations.  The  selection  of  the  attributes 
was  based  on  their  usefulness  In  distinguishing  the  categories  of  the 
taxonomy,  which  in  turn  will  serve  as  a  basis  in  developing  guidelines 
for  training  and  scenario  generation. 

The  components  and  their  attributes  are  shown  in  Figure  7-8.  While 
there  is  an  interrelationship  between  the  attributes  and  their  values 
across  components,  the  values  of  each  attribute  are  independently  as¬ 
signed  to  each  component.  For  example,  the  description  and  values  of 
the  attribute  "complexity"  need  not  be  the  same  when  it  pertains  to  mal¬ 
functions  as  when  it  pertains  to  cues.  A  description  of  the  attributes 
and  their  values  follows.  This  is  an  initial  attempt  to  describe 
relevant  attributes  and  values,  with  alterations  and  additions  to  be 
made  in  the  future,  when  the  taxonomy  is  put  to  use. 

7.3.2. 1  Situation-Dependent  Attributes.  "Situation"  refers  to  all 
factors  which  affect  the  responses  of  the  aircrew  and  the  outcome  of  an 
emergency.  This  Includes  mission  profile,  flight  phase,  weather,  time 
of  day,  communications,  distance  from  help,  and  other  relevant  factors. 

(1)  Predictabi lity.  All  components  have  "predictability"  as  an 
attribute,  and  in  each  case,  it  will  have  the  same  descrip¬ 
tion  and  the  same  values,  namely,  “mostly  predictable,  not 
predictable,  and  partially  predictable."  When  any  one  of 
the  components  Is  predictable,  it  means  that  all  the  fac¬ 
tors  that  are  relevant  to  the  emergency  can  be  described 
and  their  influence  on  the  emergency  predicted.  "Predicta¬ 
bility"  does  not  Imply  that  the  emergency  Itself  can  be 
predicted,  but  rather  that  it  is  possible  to  specify  the 
influence  of  each  situational  factor  on  the  outcome  of  the 
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FIGURE  7-8. 

COMPONENTS  ANO  ATTRIBUTES  OF  EMERGENCY  SITUATIONS 
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emergency.  For  example,  an  instrument  failure  in  VFR  con¬ 
ditions  is  a  very  different  emergency  than  in  poor  weather 
or  at  night;  in  this  case,  the  effects  of  visual  conditions 
on  instrument  failure  can  be  described.  On  the  other  hand, 
there  may  be  factors  which  will  affect  an  emergency  situa¬ 
tion  which  cannot  be  predicted  and  hence,  cannot  be  includ¬ 
ed  in  scenarios.  For  example,  meteorological  conditions 
can  produce  visual  illusions  that  cannot  be  predicted.  In 
an  accident  with  a  T-37B,  a  student  pilot  perceived  both 
fire  warning  lights  illuminated,  and  noted  smoke  billowing 
over  the  right  wing.  Later  investigation  disclosed  that 
the  student  pilot  ejected  from  a  flyable  aircraft,  possibly 
due  to  cockpit  glare,  vrtiich  appeared  to  illuminate  the  fire 
warning  lights  and  caused  an  illusion  of  smoke.  Cases  such 
as  these  are  not  likely  to  be  predicted  before  they  actual¬ 
ly  occur. 

Predictability  as  an  attribute  is  more  easily  understood 
with  respect  to  malfunctions  or  cues.  Malfunctions  that 
have  not  been  predicted  occur  relatively  frequently,  as 
evidenced  by  the  Safety  Recommendations  Bulletins  issued  by 
the  National  Transportation  Safety  Board  (NTSB).  Any 
nunber  of  malfunctions  can  occur  that  cannot  be  predicted. 
For  example,  a  fatigue  crack  in  a  tail  rotor  blade  of  a 
Sikorsky  S61L  helicopter  caused  a  35-inch  outboard  section 
of  one  of  the  tail  rotor  blades  to  separate  in  flight, 
resulting  in  a  massive  failure  in  the  tail  rotor  gear  box. 
The  helicopter  crashed  with  fatal  consequences.  The  crack 
was  not  detectable  by  visual  examination  and  the  failure 
could  not  have  been  predicted.  (The  safety  board  recaa- 
mended  that  the  airworthiness  certificate  of  the  S61  air- 
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craft  be  withdrawn  until  a  means  of  detecting  potential 
tail  rotor  blade  failures  could  be  devised  and  implemented; 
NTSB  Safety  Recommendations  A-79-25  and  A-79-26. ) 

(2)  likelihood.  This  attribute  can  be  assigned  ar\y  value 

between  0  and  1,  because  likelihood  refers  to  the  probabil¬ 
ity  that  a  particular  situation  will  occur,  and  this  can  be 
estimated  from  frequency  data.  However,  for  practical  pur¬ 
poses,  three  values  will  be  used:  high,  medium,  and  low. 
This  attribute  is  important  in  that  it  may  suggest  how  much 
training  time  should  be  devoted  to  a  certain  combination  of 
events.  Likelihood  is  also  an  attribute  that  applies  to 
all  four  components,  and  for  which  the  attribute  values 
will  remain  the  same  across  components. 

7. 3. 2. 2  Halfunction-Dependent  Attributes.  Malfunctions  refer  to  any 
physical  breakdown,  failure,  or  Irregularity  in  the  system.  The  cause 
of  the  malfunction  is  irrelevant  at  this  point,  although  eventually,  the 
taxonomy  should  be  expanded  to  include  events  preceding  the  onset  of  a 
malfunction,  since  the  identification  of  such  causal  factors  can  also 
lead  to  the  development  of  better  training  guidelines.  Attributes  of 
malfunctions  include: 

(1)  Predictability.  Same  as  above. 

(2)  Likelihood.  Same  as  above. 

(3)  Complexity.  The  complexity  of  a  malfunction  refers  to  the 
malfunction  itself,  its  physical  repercussions  on  other 
parts  of  the  system,  and  the  ease  or  difficulty  with  which 
it  can  be  described.  The  values  include  single,  compound, 


or  sequential.  A  battery  failure  would  be  an  example  of  a 
single  malfunction,  Pereas  a  total  generator  failure  would 
be  considered  a  compound  malfunction.  A  sequential  mal¬ 
function  refers  to  several  malfunctions  occurring  in  se¬ 
quence,  whether  or  not  they  are  directly  related  to  each 
other. 

(A)  Criticality.  This  is  an  Important  attribute,  which  refers 
to  the  potential  criticality  of  the  emergency  situation  in 
terms  of  its  outcome.  While  both  the  situation  and  the 
cues  can  vary  if.  criticality,  it  Is  the  type  of  malfunction 
that  is  the  major  determinant  of  an  emergency  situation's 
outcome.  Criticality  has  four  values:  minimal,  medium, 
major  and  ambiguous.  A  highly  critical  malfunction  has  the 
potential  for  a  disasterous  outcome,  with  danger  to  the  pi¬ 
lot  and/or  the  aircraft.  Medium  and  minimal  refer  to 
correspondingly  lower  outcome  criticalities.  These  values 
can  be  obtained  by  having  experts  rate  emergency  conditions 
as  was  described  in  Chapter  6.  A  situation  having  ambigu¬ 
ous  criticality  is  one  in  which  the  outcome  depends  on  a 
combination  of  factors  that  are  dependent  on  environmental 
events  (e.g.,  poor  weather  makes  an  instrument  failure  much 
more  critical  than  good  weather),  the  system  (e.g.,  the 
criticality  of  an  engine  failure  depends  on  whether  the 
aircraft  has  one,  two,  or  more  engines,  and  if  it  has  two 
engines,  whether  it  is  the  first  or  the  second  engine 
failure),  or  on  the  pilot's  behavior  (e.g.,  at  what  point 
in  time  the  emergency  is  perceived  and  dealt  with  can 
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determine  the  criticality  of  the  malfunction).  Criticali¬ 
ty,  just  as  likelihood,  may  suggest  the  amount  of  training 
that  should  be  devoted  to  a  particular  malfunction  or  com¬ 
bination  of  malfunctions. 

At  the  time  of  an  emergency  and  from  the  pilot’s  point  of  view,  the  com¬ 
plexity  and  the  criticality  of  the  malfunction  are  probably  not  as  im¬ 
portant  as  the  complexity  and  perceived  criticality  of  the  cues,  as  ex¬ 
emplified  in  the  following  abbreviated  report  of  a  minor  accident  in¬ 
volving  a  B-S2G  aircraft: 


immediately  after  liftoff,  all  engine  instruments  for 
the  right  wing  engines  began  erratic  fluctuations  and 
several  generator  circuit  breakers  opened.  During 
flap  retraction,  the  number  6  engine  fire  light 
illuminated  and  the  engine  was  shut  down.  The  fire 
light  remained  on  and  the  number  5  engine  throttle  was 
reduced  to  idle.  At  this  time,  the  number  8  engine 
fire  light  illuminated  and  the  number  8  engine  was 
shut  down.  Then,  the  number  5  engine  fire  light 
illuminated  and  the  number  7  engine  fire  light 
flickered.  Due  to  the  low  altitude  and  aircraft 
gross  weight,  the  aircrew  elected  not  to  shut  down 
the  numbers  5  and  7  engines.  Fuel  was  burned  down 
and  an  uneventful  six-engine  landing  was  made.  The 
aircraft  sustained  minor  damage. 


During  the  investigation,  it  was  discovered  that: 


Some  time  between  engine  start  through  takeoff, 
the  bleed  air  manifold  duct  assembly  failed  due 
to  tensile  overloading  of  an  undetermined  nature. 
As  a  result  of  the  released  hot  bleed  air,  wire 
bundles  in  the  right  wing  leading  edge  burned  or 
melted  resulting  in  multiple,  unrelated,  serious 
aircraft  malfunction  indications. 


Here  the  cues  were  complex  and  appeared  highly  critical,  even  though  the 
underlying  malfunction  was  simple  and  of  low  criticality.  This  suggests 
that  the  aircrew's  behavior  must  first  be  guided  by  the  cues,  and  only 
secondarily  by  their  evaluation  concerning  the  nature  of  the  malfunc¬ 
tion. 

7.3.2. 3  Cue-Dependent  Attributes.  A  cue  is  any  manifestation  in  the 
pilot's  environment  that  is  unexpected  and  that  is  perceived  through  any 
of  the  senses.  Cues  are  conceptualized  as  patterns  of  information  that 
are  described  in  terms  of  a  set  of  attributes.  Because  cues  represent 
information  concerning  the  malfunction,  several  of  the  attributes  refer 
to  the  relationship  between  the  cues  and  the  malfunction  (indirect  at¬ 
tributes),  rather  than  to  the  cues  themselves  (direct  attributes).  This 
is  an  important  aspect  of  an  emergency,  which  is  categorized  separately 
in  the  taxonomy  and  which  must  be  emphasized  during  training.  The 
interpretation  of  cues  is  not  only  a  function  of  training  however,  hut 
also  of  general  and  specific  experiences  with  the  aircraft.  Much 
research  remains  to  be  dene  before  all  their  relevant  characteristics 
can  be  included  in  scenarios  although  the  following  attributes  are  be¬ 
lieved  to  be  important: 

Direct  Attributes. 


(1)  Predictabi lity.  Same  as  above. 

(2)  likelihood.  Same  as  above. 

(3)  Complexity.  The  complexity  of  cues  Is  Interpreted  dif¬ 
ferently  than  the  complexity  of  malfunctions.  Malfunctions 
can  be  described  objectively,  whereas  cues  are  only  mean¬ 
ingful  as  they  are  perceived  and  interpreted  by  the  pilot. 


Only  two  values  are  assigned  to  the  complexity  of  cues: 
simple  and  complex,  and  they  refer  to  the  number  of  cues 
perceived  and  used  in  diagnosing  an  emergency.  The  rela¬ 
tive  salience  of  the  cues  to  each  other  is  included  under 
complexity. 

(4)  Recogni lability.  Recognizabil ity  or  salience  refers  to  the 
probability  that  a  particular  cue  will  be  perceived  and  the 
time  required  to  perceive  it.  Although  this  is  a  continu¬ 
ous  variable,  only  three  values  will  be  assigned:  cues 
have  a  high,  medium,  or  low  probability  of  being  recog¬ 
nized.  Recognizabil ity  is  a  function  of  the  perceptual 
sensitivity  of  the  operator,  as  well  as  of  the  information 
load  existing  at  the  time,  wh ich  Includes  the  complexity  of 
the  total  pattern  of  cues. 

(5)  Variability.  Variability  refers  to  the  fact  that  some  cues 
are  static,  while  others  are  dynamic;  i.e.,  they  change 
over  time.  These  are  the  only  two  values  assigned  to  this 
attribute.  For  example,  a  warning  light  is  a  static  cue, 
whereas  gauge  fluctuation  is  dynamic,  since  the  degree  of 
fluctuation  can  increase  0"  decrease  over  time. 

Indi rect  Attributes:  Cue/Mal function  Relationship. 

(1)  Relevance.  Some  cues  are  relevant  and  some  are  irrelevant 
in  diagnosing  a  malfunction;  these  are  the  two  values  of 
this  attribute. 

(2)  Diagnosticity.  While  relevance  is  an  attribute  that 
characterizes  individual  cues  with  respect  to  the  malfunc- 
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tion,  diagnosticity  refers  to  the  pattern  of  available 
cues:  the  pattern  may  or  may  not  be  diagnostic  in  identi¬ 
fying  a  malfunction.  That  is,  a  given  pattern  of  cues  may 
be  very  characteristic  of  a  malfunction,  or  it  may  be  puz¬ 
zling,  in  the  sense  that  some  of  the  cues  in  the  pattern 
suggest  one  malfunction,  while  others  suggest  a  different 
malfunction. 

(3)  Reliability.  This  attribute  is  assigned  two  values,  reli¬ 
able  and  not  reliable.  When  a  cue  is  reliable,  it  con¬ 
sistently  indicates  the  presence  of  a  particular  malfunc¬ 
tion  and  when  it  is  not  reliable,  it  may  or  may  not  suggest 
the  presence  of  a  given  malfunction. 

These  cue/malfunction  relationships  were  enumerated  in  an  attempt  to 
identify  all  possible  characteristics  that  may  be  important  in  categor¬ 
izing  emergency  situations.  In  practice,  however  (at  least  for  the  time 
being)  only  two  attributes  will  be  used  which  subsume  those  listed 
above: 

(1)  Complexity.  The  pattern  of  cues  is  complex  or  not  complex 
with  respect  to  the  malfunction. 

(2)  Conqruity.  There  is  a  congruous  or  an  incongruous  relation¬ 
ship  between  the  cues  and  the  malfunction.  If  the  rela¬ 
tionship  is  congruous,  the  cues  rather  easily  identify  the 
malfunction  (their  pattern  is  reliable),  whereas,  if  the 
relationship  is  incongruous,  they  represent  a  puzzle  as  to 
the  identity  of  the  malfunction. 
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7. 2. 2. 4  Behavior-Dependent  Attributes.  Behavior  includes  overt  ac¬ 
tions,  as  Mell  as  decision  making  and  other  types  of  cognitive 
processes.  The  attributes  that  are  important  to  the  taxonon^y  and  to  the 
development  of  training  guidelines  are  as  follows: 

(1)  Predictabl 11 ty.  Behavior  is  predictable  or  not  predict¬ 
able.  When  it  is  predictable,  the  components  of  the 
behavior  required  to  deal  with  an  emergency  can  be  speci¬ 
fied.  This  does  not  include  entire  sequences  of  behavior, 
strictly  organized  and  prescribed.  When  behavior  is  said 
to  be  predictable,  it  is  possible  to  specify,  in  general, 
which  types  of  actions  and  decisions  will  be  necessary. 

(2)  Likelihood.  This  refers  to  the  probability  that  a  sequence 
of  actions  will  be  utilized  in  response  to  an  emergency. 

The  amount  of  training  and  testing  of  such  actions  is 
determined  by  the  value  of  this  attribute  which  has  three 
values:  high,  mediun,  and  low. 

(3)  Programmabi lity.  Behavior  is  programmable  or  not  programm¬ 
able.  Programmable  behavior  Is  characteristic  of  the  ac¬ 
tions  prescribed  by  Boldface  procedures  in  that  entire  se¬ 
quences  of  actions  can  be  specified  and  trained  in  advance 
of  the  emergency.  Programmed  behavior  typically  does  not 
Include  complex  cognitive  components. 

(4)  Complexity.  Complexity  refers  specifically  to  the  complex¬ 
ity  of  the  decisions  that  are  involved  in  a  particular  si¬ 
tuation  and  has  two  values,  simple  and  complex. 


7-27 


(5)  Criticality.  Criticality,  with  respect  to  behavior,  refers 
to  the  amount  of  time  available  to  perform  an  action.  Some 
emergency  situations  are  highly  time-critical,  while  others 
are  not,  but  this  attribute  affects  the  behavior  rather 
than  the  situation  itself.  Criticality  has  three  values: 
high,  medium  and  low.  High  time-criticality  refers  to  si¬ 
tuations  in  which  only  a  few  seconds  are  available  to  make 
a  decision  or  take  an  action.  If  a  situation  has  medium 
time  criticality,  there  is  some  time  pressure  involved,  but 
there  is  still  enough  time  to  evaluate  the  situation  and 
consider  alternatives,  whereas  in  low  time-critical  situa¬ 
tions,  this  attribute  is  for  practical  purposes  insignifi¬ 
cant. 

7.4  'nitial  Training  Guidelines 

7.4.1  Theoretical  Derivations.  The  taxonomic  structure  that  has  been 
described  presents  a  number  of  attributes  that  can  be  considered  in  the 
design  of  emergency  training  programs,  and  in  particular,  emergency 
training  materials  and  mission  scenarios.  One  attribute  in  particular, 
predictability,  is  a  key  element  in  describing  or  classifying  the  com¬ 
ponents  of  an  emergency.  As  defined  earlier,  predictability  refers  to 
the  specificity  with  which  details  of  an  emergency  situation  can  be 
described  and  the  appropriate  response  behaviors  can  be  prescribed. 

Other  attributes,  of  course,  have  implications  for  training  program 
design.  However,  predictability,  because  of  its  central  role  in  the 
representative  models  presented,  will  be  explored  in  the  remainder  of 
this  section  as  one  guiding  principle  for  emergency  training  program 
development. 
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Three  major  classes  of  emergency  situations  can  be  identified,  depending 
on  the  degree  of  the  predictability  of  their  components.  The  general 
structure  of  these  situations  is  shown  in  Figure  7-9.  The  levels  of 
representation  and  the  relevant  components  for  each,  are  listed  in  the 
left-hand  columns.  For  each  level  of  representation  and  its  appropriate 
components,  the  attribute  values  are  specified  for  each  of  three  situa¬ 
tions.  The  three  situations  are  simply  labeled  “predictable,"  "partial¬ 
ly  predictable,"  and  “not  predictable,"  or  situations  1,  2,  and  3, 
respectively.  In  Figures  7-10,  7-11,  and  7-12,  the  appropriate  attri¬ 
bute  values  for  each  situation  are  shown  separately. 

Figure  7-10  defines  situation  1,  in  which  the  events,  the  behavior,  and 
the  outcomes  are  predictable.  In  general,  only  single  malfunctions  will 
fall  into  this  category,  and  their  cues  will  be  well-defined,  recogniz¬ 
able,  and  will  have  high  diagnosticity  and  reliability.  These  values 
imply  that  there  is  a  simple,  congruous  relationship  between  the  cues 
and  the  malfunction.  When  this  is  the  case,  v<*ry  little  decision-making 
is  necessary  at  the  time  the  malfunction  is  diagnosed;  at  the  most,  some 
problem  recognition  and  structuring  may  be  required.  Since  the  event 
sequence  is  well-defined  and  predictable,  the  best  decision  rule  can  be 
determined  at  the  time  the  event  sequence  is  described,  as  can  the  most 
appropriate  responses. 

The  cognitive  structure  implicit  in  this  type  of  situation  is  that  of  a 
single  template  that  contains  all  the  information  necessary  to  recognize 
the  malfunction  and  to  deal  with  it.  Thus,  the  process  is  one  of  recog¬ 
nition  and  of  matching  the  correct  template  to  the  situation.  The  im¬ 
plication  for  training  is  essentially  the  same  as  that  underlying  Bold¬ 
face  procedures.  The  entire  pattern  of  cues  must  be  trained  so 
thoroughly  that  the  correct  responses  to  it  are  immediate.  In  some 
cases,  there  may  be  some  fuzzy  boundaries  between  two  or  more  cue  pat- 
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terns,  so  that  some  training  in  problem  recognition  and  structuring  will 
be  required  to  deal  with  malfunctions  belonging  to  situation  1. 

Figure  7-11  lists  the  attribute  values  for  the  partially  predictable  si¬ 
tuations.  These  are  situations  that  can  be  foreseen,  but  for  which  de¬ 
cisions  cannot  be  rigidly  programmed  because  there  are  too  many  poten¬ 
tial  complexities  that  affect  the  decisions  and  the  actions  involved. 
Three  types  of  malfunctions  can  belong  to  situation  2:  single,  com¬ 
pound,  or  sequential.  If  a  malfunction  is  single,  it  has  to  have  ambi¬ 
guous  cues  tc  be  categorized  in  situation  2.  Ambiguous  cues  are  those 
that  suggest  either  no  spe  fic  malfunction,  or  more  than  one  malfunc¬ 
tion,  so  that  the  cue/malfunction  relationship  is  complex.  If  the  mal¬ 
functions  are  compound  or  sequential,  the  cues  can  be  either  well- 
defined  or  ambiguous. 

To  some  extent,  compound  and  sequential  malfunctions  can  be  predicted, 
but  the  number  cf  possible  combinations  is  so  great  that  it  is  not  pos¬ 
sible  to  present  all  combinations  in  a  training  course.  For  this  reason 
a  more  generalized  approach  to  decision  training  may  be  necessary.  All 
three  types  of  decision  tasks  (prob’em  structuring,  alternative  selec¬ 
tion,  complete  decision),  as  well  as  the  rules  for  selecting  the  best 
decision,  must  be  trained  for,  so  that  pilots  will  be  able  to  evaluate 
applicable  procedures  and  rules  at  the  time  of  the  emergency,  rather 
than  to  rely  on  inappropriate  or  overly  rigid  prescribed  responses. 
Desired  responses  are  predictable  in  the  sense  that  generic  situations 
can  be  devised  for  training,  but  not  programmed  to  the  level  of  detail 
of  a  specific  behavioral  sequence  which  applies  to  each  unique  situa¬ 
tion. 

The  cognitive  structure  underlying  situation  2  emergencies  consists  of 
several  templates,  or  of  representational  systems  with  overlapping  ele- 
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ments,  so  that  feature  recognition  and  integration  of  the  Information 
from  several  sources  is  necessary.  This  requires  more  active  recall 
than  the  simple  recognition  and  matching  of  situation  1  emergencies. 
Because  the  set  of  potential  situations  belonging  to  this  category  is 
very  large,  training  materials  must  be  carefully  structured  and  careful¬ 
ly  controlled  to  ensure  that  all  essential  factors  are  included,  and 
that  they  are  graduated  with  respect  to  their  difficulty.  Most  emergen¬ 
cies  that  occur  will  fall  into  this  category  since  most  accidents  that 
do  take  place  are  the  result  of  repeated  causes  with  known  precedents 
(Parker,  1978)  but  varying  in  situational  detail. 

Figure  7-12  presents  the  conceptualization  of  unpredictable  situations. 
The  malfunctions  are  unpredictable  and  probably  complex.  The  cues  may 
be  complex,  bu'  they  are  certainly  ambiguous  because  if  the  malfunction 
cannot  be  foreseen,  the  cues  will  probably  not  display  a  recognizable 
pattern.  If  the  cue  pattern  is  recognizable,  it  may  be  misleading,  so 
that  the  relationship  of  the  cues  to  the  malfunction  will  be  incongru¬ 
ous.  In  these  cases,  only  complete  decisions  (Type  3)  will  be  appropri¬ 
ate,  and  an  appropriate  decision  rule  is  to  min' mi ze  the  risk  and  to  as¬ 
sume  the  worst  possible  outcome.  No  templates  will  be  available  to  deal 
with  this  situation,  but  a  high  degree  of  Integration  between  disparate 
representational  systems  will  be  required.  Effective  responses  may  be 
compared  to  creative  problem  solving,  namely,  to  apply  old  solutions  to 
problems  that  have  nevr  been  encountered  before,  or  to  induce  the  oc¬ 
currence  of  uncommon  responses.  The  training  requirements  are  similar 
to  those  for  situation  2  emergencies,  but  they  must  emphasize  this  added 
creative  aspect--practice  in  generating  low-probability  events  and  pro¬ 
cedures.  This  emphasis  can  be  achieved  by  presenting  simulated  situa¬ 
tions  that  require  unusual  solutions  and  by  reinforcing  the  use  of  such 
solutions.  Techniques  for  eliciting  low-probabll Ity  responses  have  been 
described  by  Haltzman  (I960)  among  others. 
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7.4.2  Review  of  Training  Implications.  The  three  classes  of  emergen¬ 
cy  situations  described  above  were  categorized  according  to  their  pred¬ 
ictability  because  of  the  central  role  this  attribute  plays  in  the 
representations  and  taxonomic  structure  developed  for  aircraft  emergen¬ 
cies.  Figure  7-13  is  a  brief  overview  of  tne  three  classes  of  emergen¬ 
cies  in  terms  of  the  taxonomic  structure  together  with  training  implica¬ 
tions  far  each  class. 

For  Situation  1,  Boldface-like  training  appears  to  be  relevant,  assuming 
the  presence  of  time  and  safety  criticality.  These  emergencies  involve 
straightforward  relationships  between  cues  and  malfunctions,  information 
processing  requirements  are  low,  and  response  procedures  are  known  and 
programmable. 

For  Situation  2,  explicit  training  in  decision  techniques  is  recommended 
since  a  less  predictable  set  of  circumstances  and  responses  is  involved 
than  in  Situation  1.  Cues  are  complex  and/or  ambiguous  with  respect  to 
identifying  malfunctions.  More  information  processing  is  required,  and 
responses  can  not  be  fully  programmed  ahead  of  time.  Training  tech¬ 
niques  which  emphasize  integration  of  several  cognitive  representational 
systems  appear  to  be  recommended. 

In  Situation  3,  cues  can  be  complex,  ambiguous,  and  perhaps  misleading. 
Responses  are  not  programmable  ahead  of  time  and  extensive  deliberation 
may  be  necessary  to  diagnose  the  situation  and  develop  an  appropriate 
response.  Training  for  these  emergencies  must  address  the  ability  to 
integrate  disparate  representational  systems  and  to  account  for  low  pro¬ 
bability  events  and  relationships.  Development  of  personal  decision 
rules,  in  which  the  pilot  establishes  techniques  for  dealing  with 
manageable  approximations  of  complex  situations,  also  appears  to  be 
recommended. 
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The  present  report  describes  an  approach  to  the  application  of  theoreti¬ 
cal  models  In  the  development  of  training  requirements.  The  taxonomy 
represents  a  framework  for  combining  theoretical  Implications  Into  a  set 
of  hypotheses  relevant  to  decision  making  and  cognitive  behavior  In  em¬ 
ergency  situations.  The  three  situations  proposed  here  are  differen¬ 
tiated  on  the  basis  of  the  degree  of  predictability  of  the  types  of 
responses  necessary  to  deal  with  various  emergency  conditions.  Other 
attributes  may  also  have  differential  implications  which  need  to  be  ex¬ 
plored  and  applied  to  the  development  of  training  guidelines. 
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THIRTY  MINUTES  OVER  FLORIDA 


Case  Study 

Lieutenant  Mike  Bryant,  stationed  at  North  Island  NAS,  Squadron  VS41, 

San  Diego,  was  flying  over  Florida  on  December  9,  1978,  when  he  experi¬ 
enced  a  pending  dual  engine  failure.  He  was  flying  an  S-3  and  had  a 
student  as  a  co-pilot.  At  that  time,  Lt.  Bryant  had  approximately  2000 
hours  of  flight  time  in  S-3s  and  S-2s,  and  had  already  experienced  11 
engine  shutdowns,  which  represents  an  above  average  amount  of  experi¬ 
ence. 

Approximately  forty-five  minutes  into  the  flight,  leveling  off  at  28000 
feet,  with  an  airspeed  of  nearly  .66  mach  (normal  cruise  speed),  Lt. 
Bryant  and  his  co-pilot  were  just  preparing  to  put  on  their  oxygen 
masks--a  precautionary  measure  required  above  27000  feet  in  case  the 
large  side  canopies  depart  the  aircraft--when  they  smelled  smoke. 

Though  oxygen  masks  are  very  important  (because  at  these  altitudes  cons¬ 
cious  time  without  the  masks  is  only  a  few  seconds  in  the  event  of  a 
large  side  canopy  departure),  had  they  been  wearing  them,  they  would  not 
have  been  able  to  smell  the  smoke. 

Because  of  the  construction  of  the  bleed  air  system  in  the  S-3,  it  is 
not  uncommon  to  smell  some  smoke,  and  Lt.  Bryant  did  not  attach  too  much 
importance  to  it.  The  slight  smell  of  smoke  continued  for  a  few 
minutes;  and,  in  addition,  the  #2  engine  gauges  started  to  fluctuate  a 
bit.  Again,  this  is  not  too  uncommon.  The  probability  of  getting  a 
whiff  of  ECS  (Environmental  Control  System)  exhaust  in  the  cockpit  of 
the  S-3  is  quite  high;  it  happens  every  day,  but  usually  at  lower  alti¬ 
tudes.  It  is  equally  as  common  for  the  gauges  to  fluctuate  a  little 
^due  to  a  loose  wire,  for  example)  but  it  is  not  common  for  all  the 
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gauges  to  fluctuate  at  the  same  time.  This  was  the  first  significant 
cue  which  started  the  process  of  problem  recognition.  In  the  S-3,  there 
are  six  primary  gauges  for  each  engine,  four  tape  gauges  and  two  dial 
gauges,  one  for  the  oil  pressure  and  one  for  the  hydraulic  system. 

Thus,  while  Lt.  Bryant  was  not  particularly  worried  at  this  time,  he  be¬ 
came  aware  that  something  might  be  amiss.  There  was  a  little  more  smoke 
and  a  little  more  fluctuation  of  the  gauges.  Every  indication  of  a 
problem  was  increasing. 

All  of  a  sudden,  there  was  a  lot  more  smoke,  and  all  the  gauges  except 
for  the  oil  pressure  gauge,  were  fluctuating  wildly.  The  fluctuations 
indicated  that  something  was  wrong  in  the  core  section  of  the  engine, 
especially  since  there  was  also  smoke  in  the  cockpit.  All  the  cues  sug¬ 
gested  that  the  #2  engine  was  coming  apart.  The  fact  that  the  oil  pres¬ 
sure  gauge  was  not  fluctuating  was  not  significant;  normally,  it  is  just 
a  matter  of  time  before  it  too  begins  to  fluctuate. 

Patterns  of  gauge  abnormalities  can  vary,  yet  they  all  point  to  a  pend¬ 
ing  engine  failure.  For  example,  a  loose  oil  pump  or  a  massive  leak 
will  cause  the  oil  pressure  gauge  to  drop  off,  while  all  ether  gauges 
will  still  look  normal.  In  either  case,  if  a  decision  has  to  be  made, 
one  has  to  assume  that  such  gauge  abnormalities  indicate  a  pending  en¬ 
gine  failure.  This  is  not  a  catastrophic  problem  in  the  S-3  since  it 
has  two  engines  and  flies  equally  well  with  a  single  engine.  Little 
power  and  only  an  insignificant  amount  of  airspeed  is  lost,  which  is 
traded  off  for  altitude.  Thus,  Lt.  Bryant  shut  down  engine  #2  and 
started  the  descent  to  avoid  pressurization  problems.  This  decision 
represents  a  Type  1  decision,  consisting  primarily  of  problem  recogni¬ 
tion  and  structuring,  but  not  requiring  alternative  generation  and 
selection,  since  there  was  really  only  one  viable  option. 


Of  course,  there  was  an  alternative,  namely  not  to  shut  down  the  engine 
at  all.  However,  given  the  indications  of  smoke  and  gauge  fluctuations 
which  suggested  an  internal  compression  or  turbine  section  failure,  the 
possible  consequences  of  not  shutting  down  the  engine  were  all  at  least 
as  serious  as  shutting  it  down.  These  particular  cues  did  not  point  to 
a  specific  malfunction;  they  simply  indicated  that  something  was  wrong, 
but  given  such  general  cues,  one  must  anticipate  the  probability  of  an 
explosion,  a  fire,  or  a  flame-out.  The  probability  of  an  explosion  is 
almost  zero,  since  it  has  never  happened,  but  due  to  the  large  amount  of 
fuel  available,  there  is  a  fairly  high  probability  of  a  fire  when  a 
non-specific  engine  malfunction  occurs.  The  criticality  of  engine  fire 
is  not  as  high  in  the  S-3  as  in  the  F-4  and  the  F-14,  for  example,  be¬ 
cause  fires  can  be  isolated  more  easily  in  airplanes  with  pod-mounted 
engines,  such  as  the  S-3,  than  with  fuselage-mounted  engines,  as  in  the 
F-4  and  the  F-14.  In  either  case,  however,  a  fire  is  to  be  avoided  as 
it  can  only  cause  more  damage  to  the  engine.  The  third  possible  outcome 
is  a  flame-out  which  would  have  the  same  effect  as  an  engine  shut-down. 

It  is  also  possible  that,  given  general  cues  indicating  a  possible  en¬ 
gine  failure,  nothing  would  in  fact  happen,  but  the  general  philosophy 
in  a  case  like  this,  is  to  assume  the  worse.  Thus,  at  this  point,  the 
only  real  alternative  available  for  Lt.  Bryant  was  to  shut  down  the  en¬ 
gine.  In  general,  if  one  has  two  engines  and  something  goes  wrong  with 
one  of  them,  one  simply  shuts  it  down,  because  the  probability  that  both 
engines  will  fail  is  infinitesimal;  in  fact,  in  the  5-year  history  of 
the  S-3,  this  has  never  happened. 

After  shutting  down  one  engine,  the  standard  procedure  requires  the  pi¬ 
lot  to  lend  as  soon  as  possible.  This  is  a  Boldface  requirement  which 
will  be  followed  in  the  majority  of  cases,  although  there  can  be  excep¬ 
tions.  For  example,  if  a  pilot  is  flying  over  enemy  territory,  he  may 
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decide  not  to  land  as  soon  as  possible.  That  is,  while  the  phrase  "land 
as  soon  as  possible"  is  a  Boldface  requirement  following  well-defined 
emergencies,  it  is  subject  to  varying  interpretations,  and  in  this 
sense,  represents  a  personal  decision  rule. 

In  It.  Bryant's  case,  there  were  ro  extenuating  circumstances  which 
would  have  prevented  him  from  landing  as  soon  as  possible.  His  next  de¬ 
cision  therefore,  involved  the  evaluation  of  the  various  landing  possi¬ 
bilities.  The  landing  decision  is  a  clear  case  of  a  Type  2,  multi¬ 
attribute  decision.  There  is  no  problem  recognition  or  structuring  in¬ 
volved,  but  there  are  several  alternatives  which  have  to  be  evaluated  on 
a  number  of  different  attributes.  The  most  important  in  cases  of  emer¬ 
gencies  is  distance,  in  compliance  with  the  directive  to  land  as  soon  as 
possible,  although  there  are  other  considerations,  such  as  weather,  fa¬ 
cilities  available,  and  type  of  airfield.  The  type  of  emergency,  the 
severity  of  the  weather,  the  difference  in  the  distances  between  air¬ 
fields,  the  availability  of  arresting  gear  and  crash  crews,  are  all  fac¬ 
tors  which  must  be  weighed  against  the  distance  of  the  closest  airfield. 

For  It.  Bryant,  the  choice  was  relatively  simple.  There  were  two  Navy 
bases  and  two  Air  Force  bases  in  the  area.  Other  factors  being  equal,  a 
Navy  base  would  be  selected  over  an  Air  Force  or  a  civilian  base,  ba- 
cause  a  Navy  base  has  the  appropriate  maintenance  facilities.  It. 
Bryant's  maintenance  crew  was  at  Cecil  Field,  about  150  miles  away,  and 
was  his  preferred  alternative  with  respect  to  the  available  facilities. 
However,  there  was  a  cold  front  between  Cecil  and  the  $-3  and  Cecil  was 
experiencing  heavy  ra’ns.  The  other  Navy  base  was  at  Pensacola,  about 
100  miles  away.  Both  these  fields  were  farther  than  the  two  Air  Force 
bases:  Eglin,  about  80  miles  away,  and  Tyndall,  about  50  miles  away. 

In  this  case,  Tyndall  had  the  best  options  with  respect  to  all  factors, 
including  distance.  If  Tyndall  had  been  closed  because  of  bad  weather. 
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the  second  choice  would  have  been  Pensacwla,  a  Navy  base,  rather  than 
Eglin.  Even  though  Eglin  was  20  miles  closer,  it  is  very  large  and  has 
several  landing  areas  which  makes  it  undesirable  because  a  pilot  cannot 
know  ahead  of  time  exactly  where  he  will  be  directed  to  land.  “They  can 
vector  you  to  maybe  90  miles  away,"  according  to  Lt.  Bryant. 

Arresting  gear  was  available  at  all  four  airfields.  For  Lt.  Bryant, 
this  just  represented  an  extra  safeguard;  he  could  have  landed  without 
it.  However,  if  engine  #1  had  failed  instead  of  engine  #2,  an  arrested 
landing  would  have  been  mandatory  because  the  functioning  of  the  utility 
hydraul ics--which  includes  the  main  brake  system,  nosewheel  steering, 
and  the  landing  gear  extension--depends  on  engine  #1.  Thus,  the  landing 
decision  does  not  only  depend  on  the  characteristics  of  the  available 
landing  areas,  but  also  on  the  type  of  emergency  necessitating  a  land¬ 
ing. 

Following  the  shut-down  of  engine  #2,  Lt.  Bryant  had  declared  an  emer¬ 
gency  with  Jacksonville  Center,  the  controlling  agency,  which  also  pro¬ 
vided  him  with  the  information  concerning  the  distances  of  the  various 
airfields.  He  had  the  capability  of  calculating  these  distances  him¬ 
self,  but  obtaining  them  from  Center  is  easier  and  faster.  He  remained 
in  continued  radio  communication  with  Center  but  does  not  remember  ask¬ 
ing  for  any  further  advice  because  the  events  became  somewhat  hectic  at 
that  time.  At  approximately  9000  feet,  on  his  descent  towards  Tyndall, 
the  whole  aircraft  started  to  vibrate;  engine  #1  was  acting  up  severely 
and  appeared  to  be  failing  as  well.  There  was  a  lot  of  noise,  and  now, 
there  was  really  cause  for  concern.  The  loud,  low-pitched  rumbling 
sounds  and  the  high  rate  of  vibration  are  typical  cues  pointing  to  an 
impending  catastrophic  engine  failure,  and  were  very  different  from  the 
cues  which  had  led  to  the  shut-down  of  engine  #2  (the  fluctuation  of  the 
engine  gauges  and  the  smell  of  smoke).  Since  these  Indications  were 
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much  more  serious,  the  best  alternative  was  to  restart  engine  #2.  It. 
Bryant  tried  this  three  times,  but  without  success. 

Since  engine  #2  could  not  be  restarted,  Lt.  Bryant  had  several  options; 
he  could  try  and  land  immediately  on  the  nearest  highway,  he  could 
eject,  or  he  could  keep  flying.  Experience  and  confidence  enabled  him 
to  choose  the  latter  option,  as  long  as  the  airplane  had  enough  power 
and  enough  speed  to  keep  flying.  This  is  also  a  personal  decision  rule; 
Boldface  procedures  do  not  cover  contingencies  such  as  these.  At  this 
point,  however,  the  decision  to  keep  flying  also  implies  that  criteria 
have  to  be  defined  for  ejecting  if  it  becomes  necessary.  These  criteria 
are  highly  subjective  and  include  the  perception  of  significant  changes 
in  the  noise  level,  the  amount  of  vibration,  and  changes  in  the  engine 
gauges.  With  a  distance  of  only  20  miles  remaining  to  reach  Tyndall, 

Lt.  Bryant  decided  to  start  ejection  procedures  if  the  engines  "got  any 
worse."  What  is  important  here  is  to  know  what  the  criteria  are  and  to 
follow  through  when  the  criteria  are  reached.  Lives  may  be  lost,  not 
because  the  pilot  does  not  know  how  to  set  the  criteria,  but  because  he 
does  not  follow  through  with  his  decision,  or  because  he  changes  his 
mind  at  the  last  minute. 

As  they  approached  the  field,  Lt.  Bryant  had  to  consider  the  factors  in¬ 
volved  in  landing  his  airplane  under  these  conditions.  Air  Force  regu¬ 
lations  and  the  GCA  controllers  wanted  him  to  do  a  10-mile  straight-in, 
which  would  have  been  normal  for  the  type  of  weather  they  had.  However, 
this  is  a  radar-controlled  approach  and  Lt.  Bryant  wanted  a  visual  ap¬ 
proach  with  a  single  radar  vector  and  a  3-mile  straight-in.  There  were 
gusty  cross-winds  of  at  least  25  knots,  but  the  conditions  were  VFR. 

Lt.  Bryant  was  able  to  see  where  the  field  was  located  and  he  had  al¬ 
ready  reviewed  the  schematic  charts  of  the  airport.  Under  these  condi¬ 
tions,  a  visual  approach  did  not  present  any  danger;  on  the  contrary,  it 
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was  much  more  expeditious  because  he  could  reach  the  field  and  the 
ground  faster  than  if  he  had  had  to  wait  for  a  radar-controlled  approach 
for  which  vectors  have  to  be  calculated.  Even  though  the  type  of  ap¬ 
proach  is  generally  decided  by  ground  control,  the  pilot  always  has  the 
option  to  override  this  decision,  and  It.  Bryant's  primary  concern  was 
not  to  lose  any  time  whatsoever.  During  the  hectic  conditions  of  the 
last  20  miles,  Lt.  Bryant  and  his  co-pilot  had  followed  all  the  pro¬ 
cedures  in  the  check-list  for  a  precautionary  landing,  but  on  final  ap¬ 
proach,  realized  that  they  had  forgotten  to  dump  gas.  Since  the  $-3  was 
a  little  too  heavy  for  the  type  of  landing  selected,  gas  was  dumped  on 
final  approach  and  they  landed. 

Following  an  emergency  landing  of  this  type,  normal  procedures  requirc- 
that  the  engines  not  be  shut  off  until  the  crash  crew  arrives,  however, 
upon  arriving  at  the  field,  the  whole  cockpit  was  suddenly,  and  very  ra¬ 
pidly,  full  of  smoke.  Lt.  Bryant  shut  down  the  APU  (auxiliary  power 
unit)  and  engine  #1,  and  he  and  his  co-pilot  left  the  cockpit  head 
first.  This  much  smoke  can  be  an  indication  of  fire,  but  because  of  the 
relationship  of  the  cockpit  to  the  engines,  this  cannot  be  visually  ver¬ 
ified  and  the  best  alternative  is  to  leave  the  aircraft  as  soon  as  pos¬ 
sible.  After  verifying  that  the  engine  was  not  on  fire,  Lt.  Bryant  re¬ 
turned  to  his  airplane,  which  was  then  towed  back  to  the  hangar. 

Because  all  instructors  at  VS-41  are  qualified  as  maintenance  check  pi¬ 
lots  as  well,  Lt.  Bryant  proceeded  to  investigate  the  source  of  the 
problem.  He  contacted  his  maintenance  crew  at  Cecil  and,  by  phone, 
discovered  that  the  failure  had  occurred  m  the  ECS  turbine.  Such  a 
failure  can  cause  a  backup  in  the  bleed  air  system  (the  back  pressure 
can  overbleed  the  engines),  explaining  the  smoke  in  the  cockpit  when  the 
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air-conditioning  system  was  on.  This  was  not  a  classic  failure;  in 
fact,  the  ECS  turbine  had  never  failed  in  this  particular  way  before. 

After  Lt.  Bryant  discovered  what  had  caused  the  emergency,  he  decided  to 
fly  back  to  Cecil.  He  did  not  consider  it  a  risky  decision,  as  long  as 
the  air-conditioning  system  was  not  used. 

Analysis 

This  entire  incident,  winch  is  summarized  in  Figure  A-l,  lasted  only 
about  30  minutes.  During  that  time,  nunerous  decisions  had  to  be  made, 
only  a  few  of  which  may  be  considered  Boldface  procedures.  At  no  time, 
however,  was  there  any  real  time  pressure,  in  the  sense  that  a  decision 
had  to  be  made  within  a  few  seconds.  A  split-second  decision  would  have 
been  necessary  in  case  of  ejection.  This  <s  the  reason  that  predeter¬ 
mined  criteria  for  ejection  are  so  important;  a  pilot  has  to  know  which 
changes  in  the  pattern  of  cues  will  cause  him  to  make  the  ejection  deci¬ 
sion.  Experience  and  confidence  seem  to  be  a  requisite  for  success. 

in  Lt.  Bryant's  case,  the  most  crucial  decision  occurred  when  engine  #1 
started  to  act  up.  There  are  different  ways  in  which  an  engine  can  "act 
up,"  and  the  evaluation  is  always  very  subjective.  The  cues  have  to  do 
with  fluctuations,  noise,  smell,  feel,  torque,  vibrations,  sound.  It  is 
possible  to  distinguish  between  different  malfunctions  from  the  way  the 
engine  “acts  up,”  but  there  are  several  different  patterns  of  cues  which 
point  to  engine  failure.  Some  patterns  of  cues  are  more  predictive  of 
malfunctions  than  others. 

From  the  cues  that  he  did  have,  Lt.  Bryant  had  to  assume  that  engine  fl 
was  failing.  There  were  no  other  cues  he  could  have  used  to  disconfirm 
this  belief,  although  there  could  have  been  cues  which  would  have  con- 
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FIGURE  A-l. 

SUMMARY  OF  EVENTS  AND  DECISIONS  IN  S-3  EMERGENCY 
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FIGURE  A- I.  (CONTINUED) 

SUMMARY  OF  EVENTS  ANO  DECISIONS  IN  S-3  EMERGENCY 
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firmed  it:  if  the  engine  had  started  to  deteriorate,  the  engine  gauges 
would  have  indicated  it;  or  if  the  smoke  had  continued  and  had  not  dis¬ 
sipated,  it  would  have  represented  confirming  evidence  of  a  pending  en¬ 
gine  failure;  other  cues  could  have  included  various  warning  lights  be¬ 
ing  illuminated. 

In  retrospect,  it  was  discovered  that  no  malfunction  had  beer,  present  in 
either  engine,  but  rather,  that  the  malfunction  was  in  the  ECS  turbine. 

A  failure  or  a  hot  spot  in  the  ECS  turbine  should  have  activated  a  warn¬ 
ing  light;  the  fact  that  it  did  not  seems  to  indicate  that  the  ECS  tur¬ 
bine  simply  did  not  get  hot  enough  to  activate  the  light.  Instead,  the 
failure  generated  cues  which  were  quite  misleading. 

An  ofvious  question  is  why  engine  #2  did  not  restart.  Apparently,  this 
was  due  to  a  peculiarity  of  this  particular  failure  which  caused  an 
over-bleed  of  both  engines.  This  should  not  have  happened,  because 
there  are  valves  in  the  engines  which  should  have  by-passed  the  bleed 
air.  The  valves  should  have  opened  to  extract  the  excess  bleed  air 
overboard.  It  is  assumed  that  the  valves  could  not  handle  that  much 
bleed,  but  the  reason  for  this  is  not  yet  known.  An  engineering  invei; 
tigation  is  presently  in  progress. 

The  relationship  between  the  pending  failures  of  the  two  engines  is  in¬ 
teresting  with  respect  to  the  problem  of  relevant  cues.  The  cues  which 
suggested  that  engine  #1  was  acting  up  were  very  different  from  those 
which  had  suggested  a  failure  in  engine  #2.  The  cues  to  engine  #2  were 
the  fluctuations  of  the  gauges,  coupled  with  the  smell  of  smoke  in  the 
cockpit.  The  cues  with  respect  to  engine  #1  were  a  loud,  low-pitched 
rumbling  sound  and  a  lot  of  vibration,  which  Lt.  Bryant  associated  with 
engine  #1  only  because  engine  #2  had  been  shut  down.  There  was  a  little 
fluctuation  of  the  engine  #1  gauges,  but  this  could  easily  have  been  at- 
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tributed  to  the  vibration.  Thus,  if  both  engines  had  still  been  run¬ 
ning,  the  cues  would  still  have  indicated  a  typical  engine  failure, 
although  it  would  not  have  been  clear  which  engine  was  failing. 

Both  sets  of  cues,  which  were  very  different,  could  only  be  interpreted 
as  a  pending  engine  failure.  Because  a  jet  turbine  or  a  small  ECS  tur¬ 
bine  basically  fail  very  similarly,  as  far  as  the  sensors  are  concerned, 
one  must  assume  the  worst,  and  It.  Bryant  had  to  act  on  that  assumption. 
But  if  engine  # 2  had  in  fact  restarted  when  he  tried  it,  he  would  have 
had  to  make  another  decision,  namely,  whether  to  keep  both  engines  run¬ 
ning  or  whether  to  shut  one  of  the  engines  down.  He  would  have  shut 
down  engine  #1,  since  its  symptoms  were  much  more  serious  than  those  of 
engine  t 2. 

However,  if  engine  i 2  had  restarted  and  he  had  shut  down  engine  ll, 
everything  would  have  pointed  to  an  ECS  failure,  because  the  noise  would 
have  continued  and  he  would  have  realized  that  it  was  not  due  to  the  #1 
engine.  In  that  case,  he  would  have  restarted  engine  #1,  and  re¬ 
evaluated  his  landing  decision.  It  would  not  have  been  as  critical  to 
land  as  soon  as  possible  and  the  decision  would  probably  have  been  to 
land  as  soon  as  practical. 

Following  his  experience,  It.  Bryant  gave  a  seminar  to  other  S-3  pilots 
and  wrote  up  the  incident  for  the  Navy  URS  (Unsatisfactory  Materiel  Re¬ 
porting  System),  a  procedure  designed  to  insure  dissemination  of  infor¬ 
mation  relevant  to  all  phases  of  flying. 

To  some  extent,  he  also  formulated  a  new  personal  decision  rule, 
namely--given  a  recurrence  of  these  symptoms--to  first  turn  off  the  ECS 
so  as  to  eliminate  at  least  one  hypothesis  in  diagnosing  the  malfunc¬ 
tion.  Turning  off  the  ECS  implies  reducing  one's  altitude  first,  so  as 


to  avoid  physiological  problems,  or  alternately,  one  can  wear  an  oxygen 
mask  while  flying  without  the  CCS.  Hearing  an  oxygen  mask,  however,  im¬ 
plies  the  loss  of  important  olfactory  cues,  hence  a  reduction  in  the 
amount  of  information  available  to  decide  on  the  best  alternative. 

This  case  study  illustrates  the  complexity  of  the  problems  pilots  may  be 
faced  with  and  the  variety  of  decisions,  some  of  them  under  stressful 
conditions,  they  may  have  to  make.  By  combining  the  results  of  case 
studies  with  theoretical  analyses  of  decision  making  and  of  cognitive 
processes,  a  better  understanding  may  be  reached  of  the  optimal  behavior 
required  in  aircraft  emergency  decisions,  which  may  serve  as  a  basis  for 
developing  improved  training  materials. 
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